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Abstract 


An  initiative  between  DeTeBerkom  in  Berlin  and  the  Communication  Research  Centre  (CRC) 
in  Ottawa  was  undertaken  to  determine  realistic  resource  reservation  requirements  when 
Internet  Protocol  (IP)  telephony  applications  are  multiplexed  with  bulk  data  applications  in  an 
Asynchronous  Transfer  Mode  (ATM)  network. 

In  our  work,  we  considered  different  ways  of  bundling  voice  (i.e.  IP  telephony)  and  data 
traffic.  We  analyse  the  throughput  achieved  for  the  data  traffic  and  the  rate,  packet  loss  and 
delay  variance  for  the  voice  traffic  for  each  bundle  type. 

We  illustrate  the  specific  effects  that  different  performance  factors  such  as  Variable  Bit  Rate 
(VBR)  traffic  parameters,  the  Transmission  Control  Protocol  (TCP)  flow  control  and  send 
window  size,  network  delay,  system  scheduling  and  application  traffic  have  on  the  Quality  of 
Service  (QoS)  provision  in  an  ATM  Wide  Area  Network  (WAN)  and  Local  Area  Network 
(LAN)  environment. 

Trans-Atlantic  connections,  using  national  high-speed  test  networks  and  Teleglobes  trans- 
Atlantic  submarine  fibre  CANTAT-3  are  used  to  obtain  the  ATM  WAN  measurements.  LAN 
measurements  are  performed  using  an  ATM  LAN  testbed  at  CRC. 


The  experiments  are  performed  and  evaluated  with  the  CM  Toolset,  which  provides  for  the 
automation  of  QoS  analysis  of  selected  applications  under  different  ATM  network  environ¬ 


ments. 


Resume 


Une  etude  a  ete  effectue  conjointement  par  le  Centre  des  Recherches  sur  les  Communications 
(CRC)  a  Ottawa  et  par  DeTeBerkom  a  Berlin.  L’objectif  de  cette  etude  etait  d’exploiter  les 
ressources  du  reseau  de  Mode  de  Transmission  Asynchrone  (MTA)  de  fagon  efficace  en 
multiplexant  des  applications  de  voix  utilisant  le  protocole  Internet  avec  des  applications  de 
donnees. 

Differentes  methodes  de  multiplexage  de  la  voix  et  des  donnees  ont  ete  considerees.  Les  tests 
consistaient  a  mesurer  les  parametres  de  qualite  de  service  MTA  tels  que  la  largeur  de  bande 
atteinte  en  fonction  du  trafic  des  donnees  ainsi  que  le  debit,  la  perte  de  paquets  et  la  variation 
du  delai  du  trafic  de  la  voix  et  ce,  pour  chaque  type  de  multiplexage. 

Les  resultats  revelent  que  certains  facteurs  tels  que  les  parametres  de  trafic  du  debit  binaire 
variable,  la  dimension  de  la  fenetre  du  protocol  de  controle  de  transmission  (TCP),  le  controle 
du  debit  TCP,  les  delais  encourrus  dans  le  reseau  et  la  repartition  du  temps  de  calcul  par  le 
systeme  d'exploitation  entre  les  processus  concurrents  ont  un  effet  specifique  sur  la  qualite  de 
service  des  reseaux  MTA. 

Les  mesures  trans-Atlantic  ont  ete  obtenues  en  utilisant  des  reseaux  nationales  ATM  et  le 
cable  sous-marin  CANTAT-3  de  Teleglobe.  Les  mesures  locales  ont  ete  effectuees  au  CRC 
sur  un  reseau  local  MTA. 

Un  outil  denomme  CM  tool  a  ete  utilise  pour  effectuer  et  evaluer  les  mesures.  Cet  outil  permet 
la  selection  des  characteristiques  du  trafic  a  generer  et  T  analyse  des  resultats  obtenus. 
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Executive  Summary 


A  collaboration  between  the  Communication  Research  Centre  (CRC)  in  Ottawa  Canada  and 
DeTeBerkom  in  Berlin  Germany  was  undertaken  to  study  the  Quality  of  Service  (QoS) 
requirements  of  User  Datagram  Protocol  (UDP)  Internet  voice  traffic  when  multiplexed  with 
Transmission  Control  Protocol  (TCP)  data  traffic  over  a  trans-Atlantic  Asynchronous  Transfer 
Mode  (ATM)  Wide  Area  Network  (WAN)  and  an  ATM  Local  Area  Network  (LAN)  Variable 
Bit  Rate  (VBR)  service.  The  purpose  of  this  investigation  was  to  determine  realistic  resource 
reservation  requirements  when  Internet  Protocol  (IP)  telephony  applications  are  multiplexed 
with  bulk  data  applications. 

A  series  of  performance  measurement  tests  were  carried  out,  between  Berlin  and  Ottawa,  in 
the  March  and  April  1998  timeframe.  Local  ATM  measurements  were  performed  at  CRC 
between  June  and  August. 

For  the  purpose  of  this  joint  Canada-Germany  research  activity,  national  ATM  networks  were 
interconnected  via  the  CANTAT-3  trans-Atlantic  submarine  fibre  cable.  Teleglobe  Canada 
provided  use  of  the  CANTAT-3  submarine  fibre  cable  system.  Access  to  the  CANTAT-3,  via 
the  national  ATM  Test  Network  facilities,  was  supported  by  the  CANARIE  Test  Network 
Operations  Committee  (TNOC)  in  Canada  and  by  Deutsche  Telekom  and  DeTeBerkom  in 
Germany.  From  Pennant  Point,  Nova  Scotia,  the  CANTAT-3  submarine  cable  lands  at  Sylt 
Germany.  Permanent  Virtual  Circuit  (PVC)  links  supporting  a  VBR  QoS  class  were  set  up 
between  the  CRC  BADlab  in  Ottawa  and  DeTeBerkom  in  Berlin. 

The  research  team  also  had  access  to  a  local  ATM  testbed  at  CRC.  LAN  measurements 
provided  a  valuable  baseline  for  comparison  with  the  measurements  for  the  trans-Atlantic 
ATM  WAN.  For  the  LAN  measurements,  PVCs  supporting  a  VBR  QoS  class  were  estab¬ 
lished. 

The  experiments  focused  on  two  types  of  Internet  applications:  bulk  applications,  which  are 
sensitive  to  minimum  throughput,  and  voice  applications,  which  are  sensitive  to  delays. 
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The  measurements  were  taken  using  the  CM  toolset  which  was  developed  by  Automated 
Testing  Solutions  (ATS)  Research  &  Consulting  GmBh  in  Berlin.  It  was  used  to  measure  the 
throughput  of  the  bulk  data  applications  and  the  voice  applications  together  with  the  packet 
loss  for  the  voice  applications.  The  delay  variance  measurements  were  taken  using  the  Adtech 
AX/4000  ATM  testset  (Adt97). 

The  objective  of  the  research  was  to  explore  how  the  bandwidth  in  ATM  networks  can  be  used 
more  efficiently.  The  goal  was  to  demonstrate  that  by  reserving  network  resources,  not  only 
for  individual  connections,  but  for  groups  of  connections  and  while  maintaining  the  traffic  and 
QoS  requirement  limitations  of  each  connection  that  network  bandwidth  resources  could  be 
used  more  efficiently.  For  instance,  it  is  important  that  each  UDP  voice  connection  that  has 
been  grouped  together  to  form  a  bundle  experiences  a  minimum  packet  loss,  that  the  transmit¬ 
ting  and  receiving  rate  are  maintained  and  that  the  delay  variance  is  kept  to  a  minimum.  We 
demonstrated  that  transport  level  connections  can  be  multiplexed  or  bundled  to  improve 
resource  reservation,  QoS  provisioning  and  bandwidth  cost.  We  also  demonstrated  that  when 
connections  are  bundled  at  the  end-system,  many  factors  affect  the  performance  of  the  voice 
and  data  traffic  over  a  VBR  service.  These  include  the  type  of  traffic  as  well  as  the  network 
delay,  the  system  scheduling,  the  rate  at  which  the  information  is  transmitted,  the  TCP  window 
sizes  and  the  TCP  slowstart  and  congestion  avoidance  mechanisms. 

In  our  experiments,  transport  connections  are  grouped  together  to  form  a  bundle  and  the  traffic 
is  sent  over  an  ATM  virtual  connection.  The  results  show  that  when  bundles  are  built  using 
only  UDP  connections  the  UDP  throughput  starts  to  decrease  when  more  than  twelve  UDP 
audio  streams  are  multiplexed  on  the  WAN.  This  same  behaviour  occurs  on  the  LAN  when  ten 
UDP  audio  streams  are  bundled.  When  a  single  TCP  connection  is  bundled  with  UDP 
connections,  the  UDP  throughput  decreases  faster.  The  TCP  window  size  and  slow-start 
mechanism  have  a  direct  impact  on  the  UDP  throughput.  With  a  large  window  size,  for 
instance  50  KB,  up  to  four  UDP  sessions  can  be  multiplexed  so  that  the  required  rate  for  the 
UDP  connections  is  maintained.  By  selecting  a  smaller  window  size,  more  UDP  connections 
can  be  bundled.  Finally  when  a  bundle  includes  many  TCP  connections  the  UDP  throughput 
decreases  even  faster.  As  the  number  of  TCP  sessions  is  increased,  the  UDP  throughput 
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decreases.  With  a  large  window  size,  for  instance  50  KB,  only  two  TCP  connections  can  be 
multiplexed  with  one  UDP  connection  so  that  the  required  UDP  rate  is  maintained.  Neverthe¬ 
less  by  choosing  a  small  window  size  up  to  six  TCP  connections  can  be  bundled  with  one 
UDP  connection  without  considerably  affecting  the  throughput  of  the  UDP  connection. 
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Introduction 


The  Canada-Germany  project  "Bundling  Internet  Traffic  over  VBR  ATM  Networks  was 
undertaken  to  study  the  Quality  of  Service  (QoS)  requirements  of  User  Datagram  Protocol 
(UDP)  Internet  voice  traffic  when  multiplexed  with  Transmission  Control  Protocol  (TCP)  data 
traffic  over  a  trans-Atlantic  Asynchronous  Transfer  Mode  (ATM)  Wide  Area  Network  (WAN) 
and  an  ATM  Local  Area  Network  (LAN)  supporting  a  Variable  Bit  Rate  (VBR)  service. 
Different  ways  of  bundling  voice  (i.e.  Internet  Protocol  (IP)  telephony)  and  data  traffic  were 
considered.  This  investigation  was  designed  to  determine  realistic  resource  reservation 
requirements  when  IP  telephony  applications  are  multiplexed  with  bulk  data  transmissions 
over  an  ATM  network. 

One  of  the  primary  benefits  of  ATM  networks  is  that  they  offer  different  service  classes  to 
differentiate  between  specific  types  of  connections,  each  with  a  particular  mix  of  traffic  and 
QoS  parameters.  The  VBR  service  class  is  well-suited  for  voice  applications  where  the  delay 
requirements  are  stringent.  When  the  UDP  voice  traffic  is  multiplexed  with  TCP  data  traffic 
over  a  single  virtual  channel  connection,  the  ATM  VBR  traffic  parameters  must  be  selected 
carefully  in  order  to  provide  efficient  throughput  for  the  TCP  traffic.  The  traffic  parameters 
that  can  be  set  when  using  the  VBR  service  class  are  the  Peak  Cell  Rate  (PCR),  the  Cell  Delay 
Variation  Tolerance  (CDVT),  the  Sustainable  Cell  Rate  (SCR)  and  the  Maximum  Burst  Size 
(MBS)[Atmf96],  The  TCP  flow  control  mechanism  has  a  direct  impact  on  the  SCR  and  MBS 
and  the  traffic  parameters  must  therefore  be  set  in  a  way  to  not  degrade  the  TCP  throughput 
[Ano97][Bon96].  When  the  traffic  parameters  are  exceeded,  the  ATM  network  can  enforce  the 
traffic  contract  by  performing  traffic  policing.  Non-conforming  cells  may  be  discarded.  When 
cells  that  carry  TCP  traffic  are  discarded,  the  TCP  throughput  can  go  down  to  zero.  In  order  to 
ensure  that  cells  are  not  discarded,  traffic  shaping  is  done  at  the  source  by  limiting  the  traffic 
rate  at  the  source  to  the  selected  SCR. 

Continuous  speech  of  acceptable  quality  must  be  reconstructed  from  voice  packets  that  have 
experienced  variable  delays  at  the  sending  station  and  through  the  network.  Various  tech¬ 
niques  for  packet-loss  recovery  and  for  jitter  compensation  have  been  proposed  to  improve  the 
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quality  of  the  voice  at  the  receiver  [Perk98]  [Chen89].  The  loss,  delay  characteristics  and  the 
transmitting  and  receiving  voice  rates  are  the  voice  QoS  parameters  that  are  of  interest  and 
therefore  monitored  in  our  experiments. 

Recently  a  large-scale  measurement  infrastructure  has  been  proposed  in  the  Internet  commu¬ 
nity  that  allows  for  performance  and  QoS  management.  Features  belonging  to  such  measure¬ 
ment  infrastructure  include  tools  and  facilities  that  provide  different  kinds  of  performance  and 
traffic  analysis,  daemons  that  provide  measurement  requirements  and  data  bases  to  store  the 
results.  Automated  Testing  Solutions  (ATS)  Research  &  Consulting  GmBh  in  Berlin  is 
working  on  such  an  infrastructure.  Its  infrastructure  was  used  to  complete  our  study  of  the 
performance  of  Internet  applications  over  ATM  networks.  This  measurement  infrastructure 
includes  facilities  to  analyse  the  factors  affecting  the  Quality  of  Service  of  specific  classes  of 
applications  and  provides  for  the  automation  of  QoS  analysis  of  selected  applications  under 
different  ATM  network  infrastructures  (trans-Atlantic  ATM  and  Local  ATM). 

Teleglobe  Canada  provided  use  of  their  CANTAT-3  submarine  fibre  cable.  The  CANARIE 
Test  Network  Operations  Committee  (TNOC)  in  Canada  and  Deutsche  Telekom  and  DeTe- 
Berkom  in  Germany  supported  access  to  the  CANTAT-3  via  national  ATM  test  network 
facilities.  The  research  team  also  had  access  to  a  local  ATM  testbed  at  the  Communications 
Research  Centre  (CRC)  in  Ottawa.  LAN  measurements  provided  a  valuable  baseline  for 
comparison  with  the  measurements  for  the  trans- Atlantic  ATM  WAN. 

1  UPC  Mechanism  for  VBR 

The  traffic  parameters  that  can  be  set  when  using  the  VBR  service  class  are  the  Peak  Cell  Rate 
(PCR),  the  Cell  Delay  Variation  Tolerance  (CDVT),  the  Sustainable  Cell  Rate  (SCR)  and  the 
Maximum  Burst  Size  (MBS)  [Atmf96].  The  PCR  specifies  the  maximum  number  cells  that 
can  arrive  at  an  endpoint  per  time.  The  CDVT  controls  how  much  time  is  allowed  to  pass 
between  consecutive  cells.  The  SCR  gives  the  long-term  average  number  of  cells  that  can 
arrive  at  an  endpoint.  The  MBS  is  the  maximum  number  of  cells  that  can  arrive  on  an  endpoint 
at  the  peak  information  rate  without  violating  the  sustained  information  rate.  The  SCR  and 
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MBS  traffic  parameters  enable  the  end-system  to  describe  the  future  cell  flow  of  an  ATM 
connection  in  greater  detail  than  just  the  PCR.  If  an  end-system  is  able  to  specify  the  future 
cell  flow  in  greater  detail  than  just  the  PCR,  then  the  network  may  be  able  to  use  the  network 
resources  more  efficiently. 

An  ATM  connection  that  is  set  up  with  specified  traffic  parameters  constitutes  a  traffic 
contract  between  the  user  and  the  network.  The  network  can  enforce  the  traffic  contract  by  a 
mechanism  known  as  Usage  Parameter  Control  (UPC),  better  known  as  traffic  policing.  UPC 
is  a  set  of  algorithms  used  by  an  ATM  switch  on  the  receipt  of  cells  within  a  connection  to 
determine  whether  or  not  the  cell  stream  is  compliant  with  the  negotiated  traffic  contract.  The 
Generic  Cell  Rate  Algorithm  (GCRA)  is  used  to  define  conformance  with  respect  to  the  traffic 
contract.  For  each  cell  arrival,  the  GCRA  determines  whether  the  cell  conforms  to  the  traffic 
contract  of  the  connection.  The  UPC  function  may  implement  the  GCRA,  or  one  or  more 
equivalent  algorithms  to  enforce  conformance.  In  general  terms,  the  GCRA  is  used  to  define 
the  relationship  between  the  PCR  and  the  CDVT,  as  well  as  the  relationship  between  the  SCR 
and  the  MBS.  Traffic  sent  across  ATM  connections  that  is  controlled  by  a  UPC  is  sometimes 
shaped  using  the  GCRA.  This  ensures  that  cells  will  not  be  inadvertently  marked  as  non- 
conformant.  Traffic  shaping  can  also  be  done  in  the  traffic  source,  e.g.,  workstation. 

Conformance  for  a  real-time- VBR  (rt-VBR)  or  non-real-time- VBR  (nrt-VBR)  connection  is 
characterised  by  a  SCR  parameter  and  corresponding  MBS  in  addition  to  a  PCR  parameter  and 
corresponding  CDVT.  Rt-VBR  and  nrt-VBR  connections  are  distinguished  by  their  QoS 
parameters  and  by  the  magnitude  of  the  MBS  supported.  Larger  MBSs  are  more  typical  for 
nrt-VBR  connections.  The  QoS  parameters  are  the  Peak-to-peak  Cell  Delay  Variation  (CDV), 
the  Maximum  Cell  Transfer  Delay  (Max  CTD),  the  Mean  Cell  Transfer  Delay  (Mean  CTD) 
and  the  Cell  Loss  Ratio  (CLR).  With  rt-VBR  the  CLR,  CDV  and  Max  CTD  are  the  QoS 
parameters  of  interest  while  for  nrt-VBR  the  CLR  and  Mean  CTD  are  the  relevant  parameters. 

Upon  the  detection  of  a  non-conformant  cell,  switches  in  an  ATM  network  can  pass,  tag,  or 
discard  any  cells  that  exceed  the  configured  peak  or  sustained  information  rate.  When  the 
tagging  option  is  used,  cells  identified  by  the  UPC  function  to  be  non-conforming  are  modified 
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by  setting  the  CLP  bit  to  1 .  The  cells  that  are  tagged  will  get  discarded  further  within  the  ATM 
network  if  further  congestion  is  experienced.  When  the  discard  option  is  used,  cells  identified 
by  the  UPC  function  to  be  non-conforming  are  discarded. 

2  TCP  Window  Control 

TCP  [Jaco88],  [Post81]  was  designed  to  provide  reliable  best-effort  service  over  a  connec¬ 
tionless  network  layer  such  as  IP.  The  protocol  mechanisms  of  TCP  are  described  in  the 
following  sub-sections  along  with  the  problems  that  occur  when  trying  to  ensure  reasonable 
performance  over  an  ATM  VBR  service. 

2.1  TCP  Window  Based  Flow 

TCP  uses  a  window  based  flow  and  congestion  control  [Jaco88],  The  maximum  window  that  a 
connection  is  allowed  to  use  is  called  the  send  window.  The  size  of  the  send  window  can  be 
set  according  to  the  receiver’s  buffer,  the  user’s  throughput  requirements  or  the  network 
bandwidth  to  be  used  by  the  connection.  The  actual  window  in  TCP  is  determined  by  the 
minimum  of  the  send  window  and  the  congestion  window.  The  congestion  window  is 
dynamically  adapted  to  the  available  bandwidth-delay  product  by  the  TCP  congestion  control 
mechanism  (e.g.,  Slowstart,  Congestion  Avoidance).  TCP  sends  traffic  in  a  sliding  window  up 
to  64  KB.  The  source  has  to  wait  until  it  receives  an  acknowledgement  from  the  target  before 
sending  further  Packet  Data  Units  (PDUs).  TCP  requires  one  acknowledgement  per  PDU. 

2.2  Slowstart  and  Congestion  Avoidance 

The  purpose  of  Slowstart  and  Congestion  Avoidance  is  to  adapt  the  PDU  flow  to  the  available 
bandwidth  since  the  transport  agent  has  no  idea  of  the  available  resources  at  connection 
establishment.  Slowstart  begins  with  the  congestion  window  set  to  one  PDU.  At  the  arrival  of 
an  acknowledgement  (ACK),  the  window  slides  by  one  PDU  and  the  congestion  window  is 
incremented  by  one  PDU,  so  two  PDUs  are  sent.  This  results  in  an  exponential  increase  in  the 
number  of  transport  layer  PDUs  sent  over  one  Round  Trip  Time  (RTT).  When  the  congestion 
window  has  reached  the  Slowstart-threshold,  Congestion  Avoidance  takes  over.  A  PDU  is 
only  sent  in  response  to  an  ACK  as  the  receipt  of  an  ACK  means  that  a  PDU  has  left  the 
network  (i.e.,  the  "conservation  of  packets  principle").  Sending  PDUs  only  in  response  to 
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ACKs  adds  a  "self-clocking  property"  to  TCP.  Congestion  Avoidance  increases  the  congestion 
window  slightly  by  one  PDU  per  RTT  to  determine  if  there  is  more  bandwidth  available 
(linear  increase). 

Slowstart  takes  RTT  *  log2  (congestion  window)  seconds  [Jaco88].  Slowstart  can  cause  burst 
and  retransmission  problems  if  TCP  exceeds  the  MBS  and  the  SCR  when  trying  to  reach  the 
optimal  window.  The  TCP  throughput  can  degrade  to  almost  zero  as  the  UPC  discards  non- 
conforming  cells.  This  causes  the  situation  where  the  TCP  data  is  retransmitted  due  to 
timeouts  and  is  discarded  again  until  the  TCP  connection  is  lost  due  to  too  many  timeouts 
[Ano97].  In  our  experiments,  in  order  to  avoid  such  performance  degradation,  the  traffic  is 
shaped  at  the  source  to  ensure  that  the  UPC  mechanism  does  not  discard  any  cells. 

3  Objectives 

In  order  to  use  ATM  networks  more  efficiently,  resources  can  be  reserved  not  only  for 
individual  connections  but  also  for  groups  of  connections  [Lamo97]  [Lamo96].  The  objectives 
of  our  experiments  are  to  characterise  voice  (i.e.  IP  telephony)  and  data  traffic  based  on  their 
respective  QoS  requirements  (i.e.,  minimum  throughput,  minimum  delay  variance,  minimum 
packet  loss)  when  different  bundling  strategies  are  used.  QoS  limits  (thresholds)  are  estab¬ 
lished  for  IP  telephony  applications  and  for  bulk  data  applications  for  the  various  bundling 
experiments.  This  investigation  allows  us  to  determine  realistic  resource  reservation  require¬ 
ments  when  IP  telephony  applications  are  multiplexed  with  bulk  data  transmissions.  The 
bundles  are  built  using  different  types  of  transport  connections: 

1.  A  UDP  voice  connection  is  multiplexed  with  an  increasing  number  of  TCP  connec¬ 
tions; 

2.  A  TCP  connection  is  multiplexed  with  an  increasing  number  of  UDP  connections; 

3.  An  increasing  number  of  UDP  connections  are  multiplexed. 
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4  ATM  Testbeds 


4.1  Trans-Atlantic  Testbed  Configuration 

Classical  IP  Permanent  Virtual  Circuits  (PVCs)  supporting  VBR  service  were  established 
between  CRC  in  Ottawa  and  DeTeBerkom  in  Berlin.  The  Classical  IP  PVCs  use  Logical  Link 
Control/Subnetwork  Attachment  Point  (LLC/SNAP)  encapsulation  as  specified  in  RFC  1483 
[Hein93]  and  are  supported  per  RFC  1577  [Laub94].  The  Intemet-Protocol-to-ATM  address 
translation  table  entries  were  programmed  manually.  The  IP  traffic  destined  for  the  remote 
host  was  sent  via  the  FORE  ATM  Adapter  card,  located  in  the  Solaris  workstation  at  CRC  and 
DeTeBerkom,  on  the  specified  Virtual  Channel  Identifier  (VCI)  and  Virtual  Path  Identifier 
(VPI)  using  the  ATM  Adaptation  Layer  (AAL)  type  5.  The  Maximum  Transmission  Unit 
(MTU)  was  set  to  9180  bytes.  A  round  trip  time  delay  (RTT)  of  103  msec  was  observed 
throughout  the  measurements.  The  PCR  and  SCR  were  set  to  6  Mb/s  and  2  Mb/s.  The  CDVT 
and  MBS  were  set  to  their  default  values.  For  instance,  in  the  MainStreetXpress  36170 
switches  the  CDVT  was  set  to  250  msec  while  the  MBS  was  set  to  32  cells.  The  default  traffic 
policing  values  were  selected.  The  MainStreetXpress  36170  switches  for  example  were  set  to 
discard  any  non-conforming  cells.  The  PCR  in  the  Fore  ATM  Adapter  card  was  set  to  1594 
kbits/s.  Traffic  was  submitted  to  the  network  such  that  the  PCR  in  the  Fore  card  was  not 
exceeded.  The  IP  traffic  was  segmented/reassembled  into  ATM  cells  in  the  FORE  card  and 
sent  on  the  trans- Atlantic  link  as  shown  in  Figure  1.  A  trans- Atlantic  submarine  fibre  cable 
system,  the  CANTAT-3  [Lamo96]  interconnected  CA*net  II,  the  Canadian  national  ATM 
network,  with  Deutshe  Telekom  the  German  ATM  network. 
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Figure  1:  Trans- Atlantic  ATM  infrastructure 

4.2  Local  ATM  Testbed  Configuration 

Classical  IP  PVCs  supporting  VBR  service  were  established  in  a  ATM  LAN  at  CRC  in 
Ottawa.  The  Classical  IP  PVCs  use  LLC/SNAP  encapsulation  as  specified  in  RFC  1483  and 
are  supported  per  RFC  1577.  The  Internet-Protocol-to-ATM  address  translation  table  entries 
were  programmed  manually.  The  IP  traffic  destined  to  the  remote  host  was  sent  via  the  FORE 
ATM  Adapter  card,  located  in  the  Solaris  workstations  at  CRC,  on  the  specified  VCI  and  VPI 
using  AAL  type  5.  The  MTU  size  was  set  to  9180  bytes.  A  RTT  of  2  msec  was  observed 
throughout  the  measurements. 
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Local  ATM  Infrastructure 


CRC  LAN  ATM 


Figure  2:  Local  ATM  infrastructure 
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5  Advanced  Toolset  for  Measurement  of  ATM  Networks  -  CM 
Toolset 

5.1  CM  Toolset 

The  Protocol  Tuning  Box  (PROTB)  [Milo95],  [Milo97]  presented  at  the  Workshop  for  new 
Multimedia  Protocols  in  Salzburg,  1995,  introduced  new  aspects  for  measuring  applications 
and  protocols.  It  allows  for  instance  the  automation  of  measurements  with  different  protocol 
parameters  and  network  configurations. 

The  CM  Toolset  is  an  extension  of  the  PROTB  and  uses  an  object-oriented  approach  (based  on 
application  and  networking  components)  to  measuring  the  performance  of  application  classes 
and  their  specific  QoS.  It  is  based  on  a  remote  test  measurement  infrastructure  that  allows 
distributed  measurements  within  different  kinds  of  networking  components.  CM  toolset 
includes: 

1.  GUI  for  the  manipulation  of  remote  measurement  objects  (network  configuration, 
specification  of  application  classes  and  their  bundling,  protocol  parameter  settings); 

2.  Daemons  for  remote  execution  of  measurement  tasks  between  the  specified  objects; 

3.  Local  performance  evaluation  system  and  a  simple  data  base  to  store  measurement 
results. 

5.2  Protocol  Tuning  Box 

The  main  part  of  the  actual  CM  Toolset  version  is  the  Protocol  Tuning  Box.  It  allows  the 
automation  of  the  performance  analysis  of  application  classes  based  on  the  UDP  and  TCP 
protocols. 

5.2.1  Setting  of  Network  Architecture 

The  Protocol  Tuning  Box  provides  an  object-oriented  approach  for  specifying  measurement 
components.  The  application  class  is  defined  with  its  entities,  for  instance  data  transfer  with 
the  sending  and  receiving  entities.  The  user  assigns  a  host  address  to  each  entity  when  the 
entities  are  created.  Depending  on  the  application  class,  further  parameters  such  as  traffic 
characteristics  (distributions  to  describe  Transport  Service  Data  Units  (TSDU)  lengths  and 
interarrival  times)  and  underlying  protocol  (TCP,  UDP)  can  be  specified.  For  data  transfer  all 
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parameters  are  given  at  the  sending  entity.  The  CM  Toolset  allows  different  configurations  of 
such  application  entities,  for  example  point-to-point,  bundling  of  application  classes,  to  be 
specified.  The  following  picture  specifies  a  bi-directional  data  transfer  for  a  specific  network 
architecture: 
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Figure  3:  Bidirectional  data  transfer 

More  measurement  definition  parameters  can  be  viewed  in  Figure  3.  For  instance,  the  user  can 
give  the  number  of  measurements  for  the  specified  test  suites  in  order  to  obtain  mean  values 
and  standard  deviations  of  the  measured  statistic. 
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Furthermore,  it  is  possible  to  specify  the  transferred  data  volume  as  either  the  number  of  bytes 
or  the  number  of  TSDUs. 

5.2.2  Specification  of  Application  Classes  and  QoS  Analysis 

Application  classes  are  described  with  their  traffic  characteristics.  The  menu  of  the  Protocol 
Tuning  Box  allows  the  specification  of  distributions  for  the  TSDU  sizes  and  the  TSDU 
interarrival  times  of  applications.  The  system  will  be  enhanced  in  the  near  future  with  session 
modelling.  An  example  of  a  traffic  specification  currently  supported  by  the  CM  Toolset  is 
shown  in  Figure  4. 
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Figure  4:  Example  of  traffic  specification  using  the  CM  Toolset 

Currently  only  point-to-point  and  point-to-multipoint  application  traffic  can  be  modelled, 
which  is  enough  to  support  a  wide  area  of  application  classes  such  as  bulk  data  transfer, 
Internet  telephony,  real-time  audio  and  video. 

The  application  traffic  characteristics  are  specified  for  the  active  sender  of  the  application 
traffic.  The  QoS  measurements  of  a  specified  application  class  and  the  protocol  used  for  the 
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provision  of  its  communication  service  will  depend  on  the  specified  application  class.  Bulk 
data  transfer,  i.e.  file  transfer,  is  used  with  the  TCP  protocol  [Post81].  The  CM  Toolset 
calculates  the  throughput  for  TCP  application  classes.  For  UDP  application  classes,  such  as 
real-time  video  and  Internet  Telephony,  the  packet  loss  rate  is  evaluated. 

5.2.3  Bundling  of  Application  Classes 

The  Protocol  Tuning  Box  supports  different  approaches  for  bundling  of  application  classes. 
The  simplest  way  is  to  bundle  application  classes  with  the  same  traffic  characteristics.  For  this 
purpose  the  menu  item  “Bundling44  can  be  used. 

Bundling  of  different  application  classes  can  be  specified  by  invoking  the  “New  bundle44 
menu  item.  Figure  5  shows  an  example  for  bundling  several  application  classes  and  sessions. 
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Figure  5:  Bundling  of  application  classes  and  sessions 

5.2.4  Setting  of  Protocol  Parameters 

The  user  of  the  CM  Toolset  can  select  the  protocol  for  the  emulation  of  the  service  with  the 
menu  item  “Protocol”.  Currently,  TCP  and  UDP  protocols  are  supported.  For  TCP,  the 
window  size  parameter  is  required  (menu  item  'Window'). 
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5.2.5  Incremental  Parameter  Measurements 


The  CM  Toolset  allows  the  user  to  automate  measurements  with  varying  parameters.  This  is 
called  incremental  measurements.  From  the  menu  items  for  incremental  parameter  settings,  the 
user  can  get  the  first  value  and  the  distribution  to  calculate  and  evaluate  the  increments. 
Currently,  incremental  measurements  are  provided  for  the  window  size,  the  number  of 
bundled  connections  and  the  TSDU  size.  Figure  6  illustrates  the  settings  for  incremental 
TSDU  size  measurements: 


Figure  6:  Incremental  measurement  scenario 

The  increment  shows  the  number  of  bytes  that  will  be  added  with  each  new  measurement.  The 
increment  number  defines  the  number  of  incremental  measurements. 


6  Characterisation  of  voice  and  data  when  the  SCR  is  restricted 

The  focus  of  these  measurements  is  to  determine  efficient  strategies  for  using  ATM  resources 
efficiently  when  UDP  and  TCP  traffic  are  multiplexed  onto  one  Virtual  Channel  Connection 
(VCC).  The  emulated  applications  include  bulk  data  and  real  time  voice,  i.e.  Internet  Teleph¬ 
ony.  Traffic  is  shaped  at  the  source  by  limiting  the  peak  rate  at  the  end-system  to  that  of  the 
SCR  for  the  connection  which  is  set  to  2  Mbits/s.  This  ensures  that  the  SCR  is  not  exceeded 
and  that  cells  are  not  discarded.  The  PCR  in  the  FORE  card  is  set  to  1594  kb/s. 

The  voice  traffic  is  sent  using  the  UDP  protocol  with  a  TSDU  size  of  80  bytes  and  a  constant 
interarrival  time  of  10  ms.  The  duration  of  the  UDP  session  was  set  to  3  minutes.  Typical  bit 
rates  used  for  Internet  Telephony  vary  from  8  to  12  kb/s.  In  our  experiments,  the  worse  case 
scenario  is  selected. 


80  ^Byte^  =  8000 [Byte  /  s]=  64 [kBit  /  s] 

0.01  [s] 

The  TCP  traffic  consists  of  bulk  data  with  no  interarrival  time.  The  number  of  TSDUs  was 
calculated  such  that  the  duration  of  the  TCP  session  was  3  minutes.  Again  a  worse  case 
scenario  is  selected  for  the  bulk  data  were  no  interarrival  time  is  considered  between  TSDUs. 

The  Adtech  AX/4000  ATM  test  system  [Adt97]  was  used  to  measure  the  cell  delay  variations 
and  the  CM  Toolset  to  obtain  the  throughput  and  byte  loss  measurements  at  the  transport  level. 
The  CM  Toolset  did  not  provide  the  facility  to  obtain  the  end-to-end  delay.  Consequently,  this 
measurement  has  not  been  considered  in  this  current  study. 
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6.1  Trans-Atlantic  Measurements 

The  trans- Atlantic  measurement  were  completed  using  a  Classical  IP  PVC  supporting  VBR  as 
described  in  section  4.1.  Several  measurement  test  scenarios  in  which  TCP  and  UDP  traffic 
were  multiplexed  were  considered.  The  bundles  were  built  using  different  types  of  transport 
connections: 

•  1.  A  bulk  data  session  (TCP)  is  multiplexed  with  an  increasing  number  of  voice  sessions 
(UDP). 

2.  A  voice  session  (UDP)  is  multiplexed  with  an  increasing  number  of  bulk  data  sessions 
(TCP). 

3.  An  increasing  number  of  voice  sessions  (UDP)  are  multiplexed. 

Several  measurement  test  scenarios  in  which  TCP  and  UDP  traffic  were  multiplexed  were 
considered.  As  the  number  of  connections  increased,  the  mean  value  of  the  throuhput  was 
calculated  by  summing  the  throughput  of  each  connection  belonging  to  the  same  transport 
connection  type  and  dividing  the  result  by  the  number  of  connections.  For  instance,  when 
bundling  one  TCP  connection  with  an  increasing  number  of  UDP  connections,  the  mean  value 
of  the  UDP  throughput  was  calculated  by  summing  the  throughput  of  each  UDP  connection 
and  dividing  the  result  by  the  number  of  UDP  connections. 
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6.1 .1  Bundling  one  TCP  connection  with  an  increasing  number  of  UDP  connec 
tions 


Measurement  Scenario: 


Description 

Goal 

•  TCP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  loss  rate  vs.  number  of  UDP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, Adtech 
AX/4000 

Applications 

Bulk  data  (TCP),  64  kbps  audio  stream 
(UDP) 

Protocols 

TCP,  UDP 

TCP 

Constant  TSDU  length:  4  Kbytes, 

Traffic  Parameters 

8  Kbytes,  10  Kbytes  and  20  Kbytes 

TSDU  interarrival:  0  ms  (unconstrained 
traffic) 

Number  of  TCP  TSDUs:  variable 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kb/s) 

Number  of  UDP 
Bundles 

1,2,3,4,6,8,12,16 

Sender 

Workstation:  endor-cip 

SPARC- 10,  Solaris  2.5.1,  Berkom 

Germany 

NIC:  ForeRunner  SBA-200 

Peak  cell  rate:  1594  kb/s  (limitation 
of  the  FORE  card) 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  cell  rate:  2000  kb/s 

ATM  WAN 

VBR  PVC  connection 

PCR  =  6  Mbits/s  MBS  =  32  cells 

SCR  =  2  Mbits/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mb/s ),  T3  (45Mb/s) 

RTT 

103  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  A,  Tables  5  through  16 
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(b)  TCP  TSDU  SIZE  8  KB 
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Figure  7:  TCP  Throughput  vs.  number  of  UDP  bundles  for  different  TCP  TSDU  lengths:  (a)  4 
KB,  (b)  8  KB,  (c)  10  KB  and  (d)  20  KB  on  an  ATM  WAN 
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(b)  2  UDP  BUNDLES 
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(C)  3  UDP  BUNDLES  (d)  4  UDP  BUNDLES 
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(e)  6  UDP  BUNDLES  (1)  8  UDP  BUNDLES 


Cell  Number  Cell  Number 

Figure  10:  Cell  interarrival  variation  when  bundling  one  TCP  connection,  using  a  window  size 
of  16  KB,  with  different  number  of  UDP  bundles:  (a)  1  UDP  bundle,  (b)  2  UDP  bundles,  (c)  3 
UDP  bundles,  (d)  4  UDP  bundles,  (e)  6  UDP  bundles  and  (f)  8  UDP  bundles  on  ATM  WAN 
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(a)  1  UDP  BUNDLE 


(b)  2  UDP  BUNDLES 


Cell  Number 


o  20  40  60  80  100 

Cell  Number 


(c)  3  UDP  BUNDLES  (d)  4  UDP  BUNDLES 


Cell  Number  Cel1  Number 


(e)  6  UDP  BUNDLES  (0  8  UDP  BUNDLES 


Cell  Number  Cell  Number 


Figure  11:  Cell  interarrival  variation  when  bundling  one  TCP  connection,  using  a  window  size 
of  50  KB,  with  different  number  of  UDP  bundles:  (a)  1  UDP  bundle,  (b)  2  UDP  bundles,  (c)  3 
UDP  bundles,  (d)  4  UDP  bundles,  (e)  6  UDP  bundles  and  (f)  8  UDP  bundles  on  ATM  WAN 
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Observations  for  figure  7: 


In  Figure  7  one  can  see  the  effect  of  an  increasing  number  of  multiplexed  voice  connections 
(UDP)  on  the  bulk  data  (TCP)  throughput.  The  RTT  delay  and  the  window  size  affect  the 
throughput  of  a  TCP  connection.  When  bundling  one  TCP  connection  with  several  UDP 
connections,  the  TCP  throughput  is  also  affected  by  the  time  required  to  send  the  UDP  traffic, 
as  show  in  Figure  12.  That  is: 


TCP  _  Throughput 


Window  _  Size 
Ttcp  +  Tudp 


With  different  window  sizes,  different  levels  of  throughput  are  achieved  [Lamo96a].  It  is  clear, 
from  Figures  7(a)  through  7(d)  that  the  throughput  achieved  by  the  TCP  connection  is  greater 
when  larger  window  sizes  are  used. 


(a) 


T 


UDP 


(b) 


Figure  12  Bundling  one  TCP  connection  with  many  UDP  connections  on  an  ATM  WAN 
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From  figure  12(a)  and  -12(b),  we  can  see  that  the  TCP  throughput  is  affected  by  the  time 
required  to  transmit  a  complete  window  (Ttcp)  and  the  time  it  takes  to  send  the  UDP  informa¬ 
tion  (Tudp)  before  the  next  TCP  window  is  sent. 

The  peak  rate  of  the  FORE  card  was  set  to  1594  kb/s,  that  is,  194.58  KB/s.  With  ATM  we 
have  5  bytes  of  overhead  per  cell,  thus  the  maximum  data  throughput  is  176.22  KB/s. 

When  using  a  large  window  size  (i.e.  50  KB  or  32  KB  window  size),  the  TCP  throughput  is 
almost  160  KB/s,  which  is  close  to  the  connection’s  bandwidth  limit.  For  these  cases,  the  time 
it  takes  to  send  one  window  (Ttcp  =  280  ms  for  a  window  size  of  50  KB  and  Ttcp  =  180  ms  for 
a  window  size  of  32  KB)  is  greater  than  the  RTT  (103  ms  for  the  trans- Atlantic  link),  thus  a 
maximum  throughput  can  be  achieved. 

As  the  number  of  UDP  bundles  increases,  the  TCP  throughput  decreases  because  the  required 
time  to  transmit  the  UDP  (Tudp)  increases  as  shown  in  figure  12(a)  and  12(b). 

Observations  for  figure  8: 

The  throughput  of  each  UDP  audio  stream  is  64  kb/s  or  7.81  KB/sec.  As  we  can  see  in  Figures 
8(a)  through  8(d),  we  can  bundle  up  to  four  UDP  audio  connections  with  one  TCP  connection 
and  maintain  the  required  7.81  KB/s  for  the  UDP  connections,  independently  of  the  TCP 
TSDU  length  and  TCP  window  size  used. 

For  all  scenarios,  when  one  TCP  connection  is  bundled  with  several  UDP  audio  connections,  a 
better  UDP  throughput  performance  is  always  obtained  when  smaller  window  sizes  for  the 
TCP  connection  are  used.  This  can  be  explained  in  terms  of  the  time,  Ttcp,  required  to  send 
one  complete  TCP  window.  To  achieve  a  64  kb/s  data  rate,  each  UDP  TSDU,  80  bytes  in  size, 
is  transmitted  every  10  ms.  As  the  TCP  window  size  is  increased,  the  required  time  to  transmit 
the  window,  TTcp,  increases  as  shown  in  Figure  12.  A  large  TTcp  increases  the  delay  of  the 
UDPs,  as  they  are  required  to  wait  while  the  TCP  window  is  being  sent,  and  therefore  a  lower 
UDP  throughput  is  achieved.  When  a  small  window  size  is  used,  as  opposed  to  a  large 
window,  the  TCP  throughput  is  naturally  lower.  Thus,  more  bandwidth  is  available  on  the  link 
for  the  UDP  connections  and  this  provides  a  better  UDP  throughput  performance. 
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Observations  for  figure  9: 


One  can  notice  from  Figures  9(a)  through  9(d),  that  when  one  TCP  connection  is  bundled  with 
an  increasing  number  of  UDP  audio  streams,  there  is  a  tendency  of  increased  UDP  loss  rate  as 
the  number  of  UDP  bundles  increase.  In  most  cases,  the  loss  rate  is  less  than  0.5  %.  Packet 
loss  rates  between  1  and  10%  can  be  tolerated,  depending  on  the  manner  in  which  voice  is 
coded  and  missing  packets  masked  [Ramj94],  Therefore  in  our  scenario,  the  performance 
parameter  that  becomes  a  critical  factor  is  the  delay  encountered  by  the  UDP  TSDUs  as  they 
are  being  transmitted. 

Observations  for  figure  10  and  11: 

Figures  10  and  1 1  show  the  cell  interarrival  time,  in  milliseconds,  for  a  sequence  of  100  ATM 
cells  of  one  UDP  connection  when  an  increasing  number  of  UDP  connections  are  bundled 
with  one  TCP  connection.  All  UDP  connections  are  being  generated  with  a  TSDU  size  of  80 
bytes  every  10  ms.  The  TCP  session  was  created  using  a  TSDU  size  of  8  KB  and  a  window 
size  of  16  KB  for  Figure  10  and  50  KB  for  Figure  1 1. 


UDP  Bundles 

Mean 

Interarrival 

(ms) 

Standard 

Deviation 

1 

10.31 

17.25 

2 

9.89 

17.26 

3 

10.43 

17.88 

4 

10.61 

18.25 

6 

11.49 

15.93 

8 

11.27 

18.20 

Table  1  Mean  cell  interarrival  times  and  corresponding  standard  deviations 

Two  case  studies  have  been  considered.  Case  1:  one  TCP  session  set  to  a  TSDU  size  of  8  KB 
and  a  window  size  of  16  KB.  Case  2:  one  TCP  session  set  to  a  TSDU  size  of  8  KB  and  a 
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window  size  of  50  KB.  For  the  two  cases,  we  calculated  the  mean  interarrival  time  and  the 
standard  deviation  (Table  1  and  Table  2).  From  Table  1  it  is  clear  that  the  cell  interarrival 
times  increase  as  the  number  of  UDP  bundles  increase.  Note  also  that  the  standard  deviation 
increases  as  the  number  of  UDP  bundles  is  increased. 

These  results  show  that  when  four  UDP  connections  are  bundled  with  one  TCP  connection, 
the  cell  interarrival  time  variation  for  the  UDP  connection  is  0.61  ms  from  the  optimum  value 
of  10  ms.  When  six  and  eight  UDP  connections  are  bundled  with  one  TCP  connection,  the  cell 
interarrival  time  is  greater  than  the  expected  10  ms.  This  observation  corresponds  with  the 
observation  made  on  Figure  8,  where  it  was  shown  that  up  to  four  UDP  connections  can  be 
bundled  with  one  TCP  connection  and  the  UDP  throughput  can  still  be  maintained  to  the 
required  rate  of  7.81  KB/s. 


UDP  Bundles 

Mean 

Interarrival 

(ms) 

Standard 

Deviation 

1 

10.53 

21.75 

2 

10.59 

21.25 

3 

10.50 

22.02 

4 

10.97 

22.11 

6 

10.85 

20.22 

8 

12.85 

22.62 

Table  2  Mean  cell  interarrival  times  and  corresponding  standard  deviations 


From  Table  2  it  is  clear  that  the  cell  interarrival  times  also  increase  as  the  number  of  UDP 
bundles  is  increased.  This  same  relationship  holds  for  the  standard  deviation.  It  increases  as 
the  number  of  UDP  bundles  is  increased.  Comparing  the  results  in  Table  2  with  the  ones  in 
Table  1,  one  sees  that  the  mean  cell  interarrival  time  is  closer  to  the  expected  10  ms,  but  the 
standard  deviation  is  greater  than  that  obtained  in  Table  1.  Therefore  the  use  of  a  larger  TCP 
window  size  causes  the  interarrival  times  for  cells  carrying  the  UDP  traffic  to  increase.  More 
TCP  traffic  is  sent  on  the  TCP  connection  before  any  data  is  sent  on  a  UDP  connection. 
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6.1.2  Bundling  one  UDP  connection  with  an  increasing  number  of  TCP  connec¬ 
tions 


Measurement  Scenario: 


Description 

Goal 

•  TCP  Throughput  vs.  number  of  TCP 
bundles 

•  UDP  Throughput  vs.  number  of  TCP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, 
Adtech  AX/4000 

Applications 

Bulk  data  (TCP),  64  kb/s  audio  stream 
(UDP) 

Protocols 

TCP,  UDP 

Traffic  Parameters 

TCP 

Constant  TSDU  length:  4  Kbytes,  8 
Kbytes,  10  Kbytes  and  20  Kbytes 

TSDU  interarrival:  0  ms  (unconstrained 

traffic) 

Number  of  TSDUs:  variable 

TCP  Bundles 

1,2,3,4,6,8,10 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kb/s) 

Sender 

Workstation:  endor-cip 

SPARC-10,  Solaris  2.5.1,  Berkom 

Germany 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  2000  kb/s 

ATM  WAN 

VBR  PVC  connection 

PCR  =  6  Mbits/s  MBS  =  32  cells 

SCR  =  2  Mbits/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mbits/s  )  T3  (45  Mb/s) 

RTT 

103  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  B,  Tables  17  through  24 
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TCP  Throughput  [KB/s]  TCP  Throughput  [KB/s] 
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Figure  13:  TCP  Throughput  vs.  number  of  TCP  bundles  for  different  TCP  TSDU  lengths:  (a)  4 
KB,  (b)  8  KB,  (c)  10  KB  and  (d)  20  KB  on  an  ATM  WAN 
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(d)  TCP  TSDU  SIZE  20  KB 
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Figure  14:  UDP  Throughput  vs.  number  of  TCP  bundles  for  different  TCP  TSDU  lengths:  (a) 
4  KB,  (b)  8  KB,  (c)  10  KB  and  (d)  20  KB  on  an  ATM  WAN 


Observations  for  figure  13: 


From  Figures  13(a)  through  13(d)  we  can  see  that  when  several  TCP  connections  are  bundled 
with  one  UDP  audio  stream,  the  TCP  throughput  decreases  as  the  number  of  TCP  connections 
increases.  This  can  be  explained  in  terms  of  the  bandwidth  required  by  each  TCP  connection 
and  the  delay  incurred  by  the  UDP  connection.  As  more  TCP  connections  are  bundled,  the 
total  link  bandwidth  must  be  shared  between  the  TCP  sessions.  On  the  other  hand,  while  the 
TCP  information  is  being  sent,  the  UDP  information  must  wait  to  be  transmitted,  thus 
increasing  the  UDP  delay  and  Tudp  which  directly  affects  the  TCP  throughput,  Figure  12.  All 
these  factors  affect  the  throughput  of  each  TCP  connection.  Also,  as  the  number  of  TCP 
processes  in  the  transmitting  workstation  increases,  the  end-system  takes  longer  to  address  the 
TCPs  requests  for  data  transfer. 

Observations  for  figure  14: 

From  Figure  14(a)  through  14(d)  we  can  see  that  up  to  two  TCP  connections  can  always  be 
bundled  with  one  UDP  without  affecting  the  required  7.81  KB/s  throughput  for  the  UDP  audio 
stream.  With  the  TCP  window  size  set  to  8  or  16  KB,  the  number  of  TCP  bundles  can  be 
increased  to  three  without  affecting  the  UDP  throughput  performance.  Additionally  up  to  six 
TCP  connections  can  be  bundled  with  one  UDP  audio  stream  without  considerably  affecting 
the  UDP  throughput  performance  by  setting  the  window  size  to  8  Kbytes. 

The  reason  why  more  TCP  connections  with  a  small  window  size  can  be  bundled  with  one 
UDP  audio  stream  without  affecting  the  UDP  throughput  performance  is  that  the  TCP 
throughput  is  lower  when  using  a  smaller  window  size  than  when  a  larger  window  size  is  used. 
Because  of  this  the  total  link  bandwidth  can  be  shared  with  more  TCP  connections  without 
affecting  the  UDP  connection. 
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6.1.3  Bundling  an  increasing  number  of  UDP  connections 


Measurement  Scenario: 


Description 

Goal 

•  UDP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  loss  rate  vs.  number  of  UDP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, 
Adtech  AX/4000 

Applications 

64  kbps  audio  stream  (UDP) 

Protocols 

UDP 

Traffic  Parameters 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms. 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kbit/s) 

UDP  Bundles 

1,  2,  3, 4,  6,  8,  10,  12,  14  and  16 

Sender 

Workstation:  endor-cip 

SPARC- 10,  Solaris  2.5.1,  Berkom 

Germany 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  2000  kb/s 

ATM  WAN 

VBR  PVC  connection 

PCR  =  6  Mbits/s  MBS  =  32  cells 

SCR  =  2  Mbits/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mbits/s )  T3  (45  Mb/s) 

RTT 

103  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  C,  Table  25  and  26 
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UDP  loss  rate  [%] 


Number  of  UDP  Bundles 


Figure  15:  UDP  Throughput  vs.  number  of  UDP  bundles  on  an  ATM  WAN 


Figure  16:  UDP  loss  rate  vs.  number  of  UDP  bundles  on  an  ATM  WAN 


Observations  for  figure  15: 


In  the  WAN  measurements,  the  number  of  UDP  audio  streams  that  can  be  bundled  is  affected 
by  the  PCR  limit.  For  the  trans- Atlantic  measurements,  the  PCR  in  the  FORE  card  was  set  to 
1594  kb/s  or  194.58  KB/s.  Each  UDP  voice  session  has  a  data  rate  of  64  kb/s  or  7.8  KB/s. 
During  our  measurements,  each  UDP  voice  session  was  characterised  by  a  UDP  data  stream 
with  a  TSDU  size  of  80  bytes  and  an  interarrival  time  of  10  ms.  However  each  UDP  TSDU  is 
encapsulated  using  AAL5  and  therefore  one  ATM  cell  is  required  to  encapsulate  the  UDP 
header  and  two  more  ATM  cells  are  required  for  the  80  bytes  of  UDP  data.  As  each  UDP 
TSDU  is  sent,  three  ATM  cells  are  required  per  TSDU,  thus  the  actual  data  rate  is  300  cells/s 
or  15.53  KB/s  per  UDP  connection.  In  Figure  15  the  mean  UDP  throughput  starts  to  decrease 
when  more  than  12  UDP  audio  streams  are  sent  together.  This  can  be  explained  in  terms  of  the 
cell  rate  per  UDP  audio  stream.  Twelve  audio  streams  require  a  bandwidth  of  186.33  KB/s 
which  is  under  the  194.858  KB/s  PVC’s  bandwidth.  As  more  UDP  streams  are  bundled,  the 
PVC’s  bandwidth  is  exhausted  and  the  UDP  throughput  performance  is  affected  and  a  lower 
throughput  is  obtained. 

Observations  for  figure  16: 

For  the  WAN  measurements  up  to  16  UDP  bundles  were  multiplexed.  In  every  scenario  the 
loss  rate  was  less  than  1%.  From  Figure  16,  it  is  clear  that  there  is  an  increase  in  cell  loss  as 
the  number  of  UDP  bundles  is  increased.  This  can  be  explained  in  terms  of  the  workload  that 
is  generated  at  the  receiving  station.  As  more  UDP  connections  are  being  bundled  the 
Operating  System’s  scheduler  at  the  receiving  workstation  must  deal  with  more  UDP 
connections,  therefore  some  of  the  UDP  information  gets  discarded  by  the  Kernel  buffer 
and/or  the  ATM  device  buffer  as  it  is  not  serviced  in  time. 
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6.2  LAN  Measurements 


The  LAN  measurements  were  done  using  a  Classical  IP  PVC  supporting  VBR  as  described  in 
section  4.1.  Several  measurement  scenarios  were  tested  for  multiplexing  TCP  and  UDP  traffic. 
The  bundles  are  built  using  different  types  of  transport  connections: 

•  A  bulk  data  session  (TCP)  is  multiplexed  with  an  increasing  number  of  voice  sessions 
(UDP). 

•  A  voice  session  (UDP)  is  multiplexed  with  an  increasing  number  of  bulk  data  sessions 
(TCP). 

•  An  increasing  number  of  voice  sessions  (UDP)  are  multiplexed. 

As  in  the  WAN  measurements,  the  mean  value  of  the  throuhput  for  the  same  types  of  transport 
connections  is  calculated  by  summing  the  throughput  of  each  connection  and  dividing  the 
result  by  the  number  of  connections.  For  instance,  when  bundling  one  TCP  connection  with  an 
increasing  number  of  UDP  connections,  the  mean  value  of  the  UDP  throughput  was  calculated 
by  summing  the  throughput  of  each  UDP  connection  and  dividing  the  result  by  the  number  of 
UDP  connections. 
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6.2.1  Bundling  one  TCP  connection  with  an  increasing  number  of  UDP  connec¬ 
tions 


Measurement  Scenario: 


Description 

Goal 

•  TCP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  loss  rate  vs.  number  of  UDP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, Adtech 
AX/4000 

Applications 

Bulk  data  (TCP),  64  kb/s  audio  stream 
(UDP) 

Protocols 

TCP,  UDP 

Traffic  Parameters 

TCP 

Constant  TSDU  length:  4  Kbytes,  8 
Kbytes  and  20  Kbytes 

TSDU  interarrival:  0  ms  (unconstrained 
traffic) 

Number  of  TCP  TSDUs:  variable 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms. 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kb/s) 

UDP  Bundles 

1,2,3,4,6,8,10 

Sender 

Workstation:  lucifer-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  cell  rate:  1594  kb/s 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  cell  rate:  1594  kb/s 

ATM  LAN 

VBR  PVC  connection 

PCR  =  6  Mbits/s  MBS  =  32  cells 

SCR  =  2  Mbits/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mbits/s ),  T3  (45  Mb/s) 

RTT 

2  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  D,  Tables  28  through  35 
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TCP  Throughput  [KB/sec] 


(a)  TCP  TSDU  SIZE  4  KB  (b)  TCP  TSDU  SIZE  8  KB 


Number  of  UDP  Bundles  Number  of  UDP  Bundles 

Window  Size  (Kbytes)  8  16  -*-32  -*•  50 1  Window  Size  (Kbytes)  8  ■■••16 -*-  32 -*•  50 1 


(C)  TCP  TSDU  SIZE  20  KB 


0  I - , - - - , - I - T - 1 

0  2  4  6  8  10  12 


Number  of  UDP  Bundles 
Window  Size  (Kbytes)  j  — « —  8  ••«•■  16  -*-  32  -*-•  5o] 

Figure  17:  TCP  Throughput  vs.  number  of  UDP  bundles  for  different  TCP  TSDU  lengths:  (a) 
4  KB,  (b)  8  KB  and  (c)  20  KB  on  an  ATM  LAN 
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UDP  Throughput  [KB/s] 


(a)  TCP  TSDU  SIZE  4  KB 


(b)  TCP  TSDU  SIZE  8  KB 


02468  10  12  02468  10 

Number  of  UDP  Bundles  Number  of  UDP  Bundles  . 

Window  Size  (Kbytes)  |-*-8  16  -*-32  50|  Window  Size  (Kbytes)  |-*-8  16  -*-  32  -*  50 


(c)  TCP  TSDU  SIZE  20  KB 


Number  of  UDP  Bundles 
Window  Size  (Kbytes)  j— ♦-•8  16  -*-  32—*- -  50 


Figure  18:  UDP  Throughput  vs.  number  of  UDP  bundles  for  different  TCP  TSDU  lengths:  (a) 

4  KB,  (b)  8  KB  and  (c)  20  KB  on  an  ATM  LAN 


Cell  Interarrival  (ms)  Cell  Interarrival  (ms) 


(a)  1  UDP  BUNDLE 


(b)  2  UDP  BUNDLES 


Cell  Number  Cell  Number 


(C)  3  UDP  BUNDLES  (d)  4  UDP  BUNDLES 


Cell  Number  Cell  Number 


(e)  6  UDP  BUNDLES  W  8  UDP  BUNDLES 


Cell  Number  Cell  Number 


Figure  20:  Cell  interarrival  time  variation  when  bundling  one  TCP  connection,  using  a 
window  size  16  KB  with  different  number  of  UDP  bundles:  (a)  1  UDP  bundle,  (b)  2  UDP 
bundles,  (c)  3  UDP  bundles,  (d)  4  UDP  bundles,  (e)  6  UDP  bundles,  (f)  8  UDP  bundles  on  an 

an  ATM  LAN 
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Cell  Interarrival  (ms)  Cell  Interarrival  (ms)  Cell  Interarrival  (ms) 


(a)  1  UDP  BUNDLE 


(b)  2  UDP  BUNDLES 


Cell  Number  Cell  Number 


(c)  3  UDP  BUNDLES  (d)  4  UDP  BUNDLES 


Cell  Number  Cell  Number 


(e)  6  UDP  BUNDLES 


(f)  8  UDP  BUNDLES 


Cell  Number 


Ceil  Number 


Figure  21:  Cell  interarrival  time  variation  when  bundling  one  TCP  connection,  using  a 
window  size  50  KB  with  different  number  of  UDP  bundles:  (a)  1  UDP  bundle,  (b)  2  UDP 
bundles,  (c)  3  UDP  bundles,  (d)  4  UDP  bundles,  (e)  6  UDP  bundles,  (f)  8  UDP  bundles  on  an 

ATM  LAN 
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Observations  for  figure  17: 


As  in  the  WAN,  an  increasing  number  of  multiplexed  UDP  voice  connections  results  in  an 
decrease  in  the  TCP  throughput  (Figure  17).  However,  in  the  LAN  case,  two  distinct  behav¬ 
iours  have  been  distinguished.  The  first  behaviour  occurs  when  less  than  four  UDP  sessions 
are  bundled  with  a  TCP  session.  An  identical  TCP  throughput  is  reached  irrespective  of  the 
window  sizes  (16  KB,  32  KB,  50  KB)  or  the  TCP  TSDU  sizes  (4  KB,  8  KB  and  20  KB).  The 
second  behaviour,  which  occurs  when  more  than  four  UDP  sessions  are  bundled  with  a  TCP 
session,  shows  different  levels  of  throughput  achieved  for  different  window  sizes,  regardless 
of  the  TCP  TSDU  sizes. 


Figure  22:  Bundling  one  TCP  connection  with  many  UDP  connections  on  an  ATM  LAN 
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The  peak  rate  of  the  FORE  card  was  set  to  1594  kb/s;  that  is,  194.58  KB/s.In  an  ATM  cell 
there  is  5  bytes  of  overhead.  Therefore  the  maximum  data  throughput  is  176.22  KB/s. 

The  link  is  fully  utilised  on  the  LAN  when  only  the  TCP  traffic  is  present  (see  Figure  22(a)) 
because  of  the  very  small  RTT.  If  we  consider  only  TCP  traffic,  the  throughput  can  be 
calculated  in  the  following  way: 


™  ,  ,  Window  _  Size 

TCP  _  Throughput  - - 

T TCP 

Therefore  when  using  a  window  size  of  50  KB,  the  time  to  send  the  window  is  Ttcp  =  280  ms. 
For  a  window  size  of  16  KB,  it  takes  =  90  ms  to  send  the  window.  For  all  the  window  sizes 
considered,  the  time  it  takes  to  send  the  window  greatly  exceeds  the  RTT  delay  (2  ms  for  the 
LAN  link),  therefore  the  pipe  is  always  full.  The  source  scheduler  is  always  busy  sending  TCP 
traffic.  However,  since  UDP  sessions  are  bundled  with  a  TCP  session,  the  source  scheduler 
has  to  take  into  consideration  the  additional  UDP  traffic.  In  our  system,  the  UDP  traffic  is 
transmitted  just  after  sending  a  whole  window  (see  Figure  22(b)).  However  it  should  be  noted 
that  the  TCP  traffic,  when  using  a  window  size  of  8  KB  with  TCP  TSDU  sizes  of  4  KB  and  8 
KB,  presents  a  different  behaviour  (see  Figure  17(a)  and  Figure  17(b)).  The  throughput 
achieved  is  less  in  the  other  scenarios  due  to  the  extra  processing  overhead  that  it  is  generated 
when  the  TCP  TSDU  sizes  are  smaller  than  the  window  size. 

When  more  than  four  UDP  connections  are  bundled  with  a  TCP  connection,  the  effects  of  the 
window  size  become  apparent.  As  the  number  UDP  connections  increases,  a  larger  gap 
between  the  TCP  “send”  operations  is  created  and  the  effect  of  different  window  sizes  become 
more  significant.  As  the  TCP  window  size  gets  larger,  more  TCP  information  is  sent  and  the 
TCP  throughput  achieved  becomes  greater.  The  small  RTT  does  not  favour  the  bundling  of 
UDP  sessions  when  they  exceed  a  certain  number  because  the  window  size  influences  the  TCP 
and  UDP  throughput. 
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Observations  for  figure  18: 


As  in  the  WAN  case,  the  throughput  of  each  UDP  audio  stream  is  64  kb/s  or  7.81  KB/s.  From 
Figures  18(a)  through  18(c),  one  can  notice  that  four  UDP  audio  connections  can  be  bundle 
with  one  TCP  connection  without  affecting  the  throughput  of  the  UDP  traffic,  independent  of 
the  TCP  TSDU  length  and  TCP  window  size  used. 

In  addition  as  already  observed  in  the  WAN  case,  when  one  TCP  connection  is  bundled  with 
several  UDP  audio  connections,  a  better  UDP  throughput  is  obtained  when  smaller  window 
sizes  are  used.  This  situation  can  be  explained  by  the  fact  that  the  sender  has  more  network 
bandwidth  available  for  the  UPD  connections  when  smaller  window  sizes  are  used. 

However,  it  should  be  noted  that  in  the  LAN  results,  the  TCP  TSDU  size  does  not  affect  the 
UDP  throughput  for  any  number  of  UDP  sessions.  The  reason  as  mentioned  before  (see 
observation  for  Figure  17)  is  the  fact  that  the  pipe  gets  full  and  the  only  factor  which  affects 
the  UDP  throughput  is  the  TCP  window  size. 

If  we  compare  the  TCP  and  the  UDP  throughput,  we  can  notice  complementary  results.  When 
larger  windows  are  used,  we  get  a  better  TCP  throughput  and  a  diminished  UDP  throughput. 
When  smaller  windows  are  used  the  opposite  situation  occurs.  This  is  a  direct  result  of  the 
small  RTT  in  the  LAN  that  leads  to  a  full  pipe  which  is  in  turn  governed  by  the  window  size 
parameter. 


Observations  for  figure  19: 

One  can  notice,  as  in  the  WAN  case,  that  when  one  TCP  connection  is  bundled  with  an 
increasing  number  of  UDP  audio  streams,  there  is  an  increase  of  UDP  loss  rate  as  the  number 
of  UDP  bundles  increases  (Figures  19  (a)  through  19  (d)).  In  most  cases,  the  loss  rate  is  less 
than  0.25%.  Although  these  loss  rates  are  acceptable,  the  delay  characteristics  also  have  to  be 
considered. 
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Observations  for  figure  20  and  21: 


As  in  the  WAN  case,  in  order  to  calculate  the  cell  inter-arrival  times,  we  have  considered  a 
sequence  of  100  ATM  cells  from  one  UDP  connection  when  several  UDP  connections  are 
bundled  with  one  TCP  connection.  All  UDP  connections  have  been  generated  using  a  TSDU 
size  of  80  bytes  and  an  interarrival  time  of  10  ms.  Two  case  studies  have  been  considered. 
Case  1:  one  TCP  session  with  a  TSDU  size  of  8  KB  and  a  window  size  of  16  KB.  Case  2:  one 
TCP  session  with  a  TSDU  size  of  8  KB  and  a  window  size  of  50  KB.  For  the  two  cases,  we 
calculated  the  mean  interarrival  time  and  the  standard  deviation  (Table  3  and  Table  4). 


Mean  Inter- 

Standard 

UDP  Bundles 

arrival  (ms) 

Deviation 

1 

10.66 

20.81 

2 

10.25 

20.46 

3 

10.57 

19.68 

4 

11.07 

20.50 

6 

12.15 

16.72 

8 

12.08 

19.42 

Table  3:  Mean  cell  interarrival  time  and  standard  deviation  for  the  Case  1 


UDP  Bundles 

Mean  Inter¬ 
arrival  (ms) 

Standard 

Deviation 

1 

13.00 

28.67 

2 

13.44 

27.92 

3 

13.83 

28.65 

4 

14.34 

26.71 

6 

14.79 

35.27 

8 

16.40 

40.53 

Table  4:  Mean  cell  interarrival  time  and  standard  deviation  for  the  Case  2 
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From  these  tables,  the  cell  interarrival  time,  as  well  as  the  standard  deviation,  increases  as  the 
number  of  UDP  connections  increases.  In  the  first  case,  the  cell  interarrival  delays,  which  are 
of  a  few  milliseconds  greater  than  the  expected  10  ms,  are  considered  acceptable  [Ramj94]. 
However,  the  large  standard  deviations  indicate  that  some  cells  are  arriving  much  later  than 
the  expected  10  ms.  This  statement  can  be  confirmed  by  examining  Figure  20  which  shows 
that  some  cells  have  a  delay  of  50  or  60  ms  and  even  reach  a  delay  of  1 15  ms.  These  delays 
might  be  caused  by  network  devices  or  perhaps  I/O  bottelnecks  at  the  workstations  or  both. 

The  standard  deviations,  which  reflect  the  cell  delay  variation,  may  not  be  considered 
acceptable  and  therefore  should  be  adjusted  to  meet  the  delay  requirements  of  a  voice  stream. 
One  solution  would  be  to  ensure  that  the  sending  workstation  discards  any  cells  that  have  been 
waiting  too  long  for  transmission  if  the  I/O  at  the  source  is  causing  a  bottelneck.  One  could 
also  use  a  buffer  at  the  receiver,  which  would  work  as  a  regulator  to  smooth  out  the  variations. 
The  size  of  this  buffer  would  depend  on  the  interarrival  of  UDP  cells  at  the  receiver  and  on  the 
end-to-end-delay  [Perk98]. 

For  larger  window  sizes,  such  us  the  50  KB,  the  mean  cell  interarrival  times  as  well  as  the 
standard  deviations  are  larger.  This  is  due  to  the  fact  that  more  information  is  sent  in  each  TCP 
window  and  therefore  the  UDP  connections  must  wait  longer  to  be  processed.  The  same  buffer 
solution,  as  detailed  above,  should  be  applied  to  meet  the  delay  requirements  of  the  UDP  voice 
traffic. 

If  the  results  obtained  in  the  LAN  (Table  3  and  4)  are  compared  to  those  obtained  in  the  WAN 
(Table  1  and  2),  one  can  notice  that  the  delay  variances  are  more  important  in  the  former  case. 
The  small  RTT  (2  ms)  in  the  LAN,  compared  to  the  much  larger  one  (103  ms)  in  the  WAN, 
has  the  effect  of  saturating  the  receiving  end-system  faster. 
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6.2.2  Bundling  one  UDP  connection  with  an  increasing  number  of  TCP  conneo 
tions 


Measurement  Scenario: 


Description 

Goal 

•  TCP  Throughput  vs.  number  of  TCP 
bundles 

•  UDP  Throughput  vs.  number  of  TCP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, 
Adtech  AX/4000 

Applications 

Bulk  data  (TCP),  64  kb/s  audio  stream 
(UDP) 

Protocols 

TCP,  UDP 

Traffic  Parameters 

TCP 

Constant  TSDU  length:  4  Kbytes,  8 
Kbytes  and  20  Kbytes 

TSDU  interarrival:  0  ms  (unconstrained 
traffic) 

Number  of  TSDUs:  variable 

TCP  Bundles 

1,2,3,4,6,8,10 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kb/s) 

Sender 

Workstation:  lucifer-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

ATM  WAN 

VBR  PVC  connection 

PCR  =  6  Mb/s  MBS  =  32  cells 

SCR  =  2  Mb/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mbits/s  )  T3  (45  Mb/s) 

RTT 

2  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  E,  Tables  36  through  41 

46 


TCP  Throughput  [KB/sec] 


(a)  TCP  TSDU  SIZE  4  KB 


(b)  TCP  TSDU  SIZE  8  KB 


Nurrter  of  TCP  Bundles  Nunnber  of  TCP  Bundles 

Window  Size  (Kbytes)  |  — 8  ■  ■  •  16 -'•»<- -32-*;  50 1  Vtfndow  Size  (Kbytes)  8  16-*- 32-* ;  50| 


(C)  TCP  TSDU  SIZE  20  KB 


Number  of  TCP  Bundles 
Window  Size  (Kbytes)  8  ■  - 16  32  -*■  50 1 


Figure  23:  TCP  Throughput  vs.  number  of  TCP  bundles  for  different  TCP  TSDU  lengths:  (a)  4 

KB,  (b)  8  KB  and  (c)  20  KB  on  an  ATM  LAN 


UDP  Throughput  [KB/s] 


(a)  TCP  TSDU  SIZE  4  KB 


(b)  TCP  TSDU  SIZE  8  KB 


Number  of  TCP  Bundles  Number  of  TCP  Bundles 

Window  Size  (Kbytes)  !“*~8 16  -*-32-#-  5o|  Window  Size  (Kbytes)  [  8  ■■  16 -*--32 -*■  5o] 


(c)  TCP  TSDU  SIZE  20  KB 


Number  of  TCP  Bundles 
Window  Size  (Kbytes)  [  — 8  16  -*-32- *50 


Figure  24:  UDP  Throughput  vs.  number  of  TCP  bundles  for  different  TCP  TSDU  lengths:  (a) 

4  KB,  (b)  8  KB  and  (c)  20  KB  on  an  ATM  LAN 
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Observations  for  figure  23: 


As  in  the  WAN  case,  when  several  TCP  connections  are  bundled  with  one  UDP  audio  stream, 
the  mean  TCP  throughput  decreases  as  the  number  of  TCP  connection  increases  (Figures  23(a) 
through  23(c)).  This  can  be  explained  in  terms  of  the  bandwidth  required  by  each  of  the  TCP 
and  UDP  connections  (see  “Observation  on  Figure  13”).  However,  in  the  LAN  case,  the 
curves  are  very  similar  because  the  link  is  fully  utilised  for  all  TCP  TSDU  sizes  considered 
due  to  the  very  small  RTT  (2  ms).  However,  when  many  TCP  sessions  are  bundled  with  one 
UDP  session,  the  window  size  does  not  affect  the  TCP  throughput.  Recall  that  when  many 
UDP  sessions  were  bundled  with  one  TCP  session  (Section  6.2.1),  the  window  size  did  not 
have  an  impact  on  the  TCP  throughput.  This  can  be  explained  by  the  fact  that  the  link  becomes 
fully  utilised  sooner  when  many  TCP  connections  are  bundled  and  the  influence  of  the 
window  size  is  therefore  reduced. 

Observations  for  figure  24: 

Note  that,  just  as  in  the  WAN  case,  up  to  two  TCP  connections  can  always  be  bundled  with 
one  UDP  connection  without  affecting  the  required  7.8  KB/s  throughput  for  the  UDP  audio 
stream.  When  the  TCP  window  size  is  set  to  8  or  16  KB,  the  number  of  TCP  bundles  can  be 
increased  to  ten  and  six  bundles,  respectively,  without  affecting  the  UDP  connection’s 
throughput.  (Figure  24(a)  through  24(c)).  The  same  reasoning,  as  in  the  WAN  case,  can  be 
applied  to  the  LAN  case  to  explain  the  behaviour  of  the  UDP  throughput  performance:  smaller 
TCP  window  sizes  yeld  lower  TCP  throughput  therefore  leaving  more  bandwidth  for  the  UDP 
audio  stream. 

If  the  results  obtained  on  the  LAN  are  compared  with  those  obtained  for  the  WAN,  it  is  clear 
that  the  LAN  case  shows  a  better  performance  regarding  the  number  of  TCP  bundles.  For 
instance,  with  a  window  size  of  8  KB,  a  maximum  of  ten  and  six  TCP  connections,  in  the 
LAN  and  WAN,  respectively,  can  be  added  to  the  UDP  traffic  without  affecting  the  UDP 
connection’s  performance.  This  difference  can  be  attributed  to  the  fact  that  the  ACKs  arrive 
much  faster  on  the  LAN  than  on  the  WAN  and  as  a  result  the  end-system  is  able  to  process 
more  UDP  traffic. 


49 


6.2.3  Bundling  an  increasing  number  of  UDP  connections 


Measurement  Scenario: 


Description 

Goal 

•  UDP  Throughput  vs.  number  of  UDP 
bundles 

•  UDP  loss  rate  vs.  number  of  UDP 
bundles 

Test  tool 

CM-Toolset,  Protocol  Tuning  Box, 
Adtech  AX/4000 

Applications 

64  kb/s  audio  stream  (UDP) 

Protocols 

UDP 

Traffic  Parameters 

UDP 

Constant  TSDU  length:  80  bytes 

TSDU  interarrival:  10  ms 

Number  of  TSDUs:  18000  (3  minutes 
duration  at  64  kb/s) 

UDP  Bundles 

1,  2,  3, 4,  6,  8,  10,  12,  14,  16,  18,  20,  26 

Sender 

Workstation:  lucifer-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

Receiver 

Workstation:  fred-cip 

SPARC- 10,  Solaris  2.4  ,  CRC  Canada 

NIC:  ForeRunner  SBA-200 

Peak  Cell  Rate:  1594  kb/s 

ATM  WAN 

VBR  PVC  connection 

PCR  =  6  Mb/s  MBS  =  32  cells 

SCR  =  2  Mb/s  CDVT  =  250  msec 

ATM  Link 

OC-3c  (  155.52  Mbits/s  )  T3  (45  Mb/s) 

RTT 

2  ms 

Adaptation  Layer 

ATM  AAL  5,  MTU  9180  Bytes 

Details 

Data 

See  Appendix  F,  Table  38  and  39 
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Observations  for  figure  25: 


As  in  the  WAN  measurements,  the  PCR  in  the  FORE  card  was  set  to  1594  Kb/s  (or  194.58 
KB/s)  and  each  UDP  session  was  characterised  by  a  TSDU  size  of  80  bytes  and  an  interarrival 
time  of  10  ms.  In  the  experiments,  each  TSDU  requires  three  ATM  cells  in  order  to  be  sent. 
Therefore,  the  effective  data  rate  is  300  cells/s  or  15.53  KB/s  for  each  UDP  connection.  In 
Figure  25,  one  can  notice  that  the  mean  UDP  throughput  starts  to  decrease  when  more  than  10 
UDP  audio  streams  are  bundled. 

In  the  WAN  measurements,  twelve  UDP  sessions  were  bundled  without  a  decrease  in  the 
mean  UDP  throughput.  Therefore,  when  more  than  ten  audio  streams  are  bundled  (equivalent 
to  155.3Kb/s)  with  the  TCP  stream,  the  receiver’s  workstation  gets  overloaded.  In  this  case, 
the  bottleneck  becomes  the  workstation,  instead  of  the  PCR  set  in  the  FORE  card  as  stated  in 
the  WAN  measurements  (see  “Observations  for  Figure  13”).  This  is  a  result  of  the  much 
smaller  RTT  in  the  LAN. 


Observations  for  figure  26: 

As  seen  in  the  WAN  measurements,  an  increase  in  the  number  of  UDP  bundles,  results  in  an 
increase  in  the  byte  loss  rate  (Figure  26,)  even  though  the  overall  loss  rate  did  not  exceed  1%. 
As  in  the  WAN  case,  the  receiving  station’s  workload  can  be  attributed  to  the  byte  loss  rate 
behaviour  (see  “Observations  for  Figure  14”). 


7  Conclusion 


When  bundles  are  built  using  only  UDP  connections,  the  mean  UDP  throughput  starts  to 
decrease  when  more  than  twelve  UDP  audio  streams  are  multiplexed  on  the  WAN  and  when 
ten  UDP  audio  streams  are  multiplexed  on  the  LAN.  As  soon  as  a  bundle  includes  a  TCP 
connection  the  UDP  throughput  decreases  faster.  The  TCP  window  size  and  the  TCP  slowstart 
and  congestion  avoidance  mechanisms  have  a  direct  impact  on  the  UDP  throughput.  With  a 
large  window  size,  for  example  50  KB,  up  to  four  UDP  sessions  can  be  multiplexed,  in  both 
the  LAN  and  WAN  case,  and  still  maintain  the  required  rate  for  the  UDP  connections.  By 
selecting  a  smaller  TCP  window  size,  more  UDP  connections  can  be  bundled  while  maintain¬ 
ing  the  required  UDP  throughput.  Finally,  when  a  bundle  includes  many  TCP  connections,  the 
UDP  throughput  decreases  even  faster.  As  the  number  of  TCP  sessions  is  increased,  the  UDP 
throughput  decreases.  With  a  large  window  size,  50  KB,  at  most  two  TCP  connections  can  be 
multiplexed  with  one  UDP  connection  while  still  maintaining  the  required  UDP’s  connection 
rate.  Nevertheless,  choosing  a  small  window  size  permits  up  to  six  TCP  connections  to  be 
bundled  with  one  UDP  connection  without  considerably  affecting  the  UDP  connection’s 
throughput  performance.  Another  parameter  that  influenced  the  TCP  and  the  UDP  throughput 
throughout  our  measurements  was  the  RTT  delay  of  the  network.  We  observed  that  the  smaller 
the  RTT,  the  better  the  UDP  throughput  when  one  UDP  session  is  bundled  with  many  TCP 
sessions.  This  is  due  to  the  fact  that  the  TCP  acknowledgements  are  received  faster  and 
therefore  allows  the  source  end-system  to  process  the  UDP  traffic  sooner.  On  the  other-hand, 
the  delay  variations  observed  on  the  LAN  are  more  pronounced  that  in  the  WAN  due  to  the 
smaller  RTT  which  has  the  effect  of  causing  saturation  at  the  end-system  sooner. 


8  Abbreviations  of  Technical  Terms 


AAL 

ACK 

ATM 

CDV 

CDVT 

CLR 

CTD 

GCRA 

GUI 

IP 

LAN 

LLC 

Max  CTD 

MBS 

MTU 

Mean  CTD 
NIC 

nrt-VBR 

PCR 

PDU 

PVC 

QoS 

rt-VBR 

RTT 

SCR 

SNAP 

TCP 

TSDU 

UDP  . 

UPC 


ATM  Adaptation  Layer 
acknowledgement 
Asynchronous  Transfer  Mode 
Cell  Delay  Variation 
Cell  Delay  Variation  Tolerance 
Cell  Loss  Ratio 
Cell  Transfer  Delay 
Generic  Cell  Rate  Algorithm 
Graphical  User  Interface 
Internet  Protocol 
Local  Area  Network 
Logical  Link  Control 
Maximum  Cell  Transfer  Delay 
Maximum  Burst  Size 
Maximum  Transport  Unit 
Mean  Cell  Transfer  Delay 
Network  Interface  Card 
non-real-time- Variable  Bit  Rate 
Peak  Cell  Rate 
Packet  Data  Unit 
Permanent  Virtual  Circuit 
Quality  of  Service 
real-time- Variable  Bit  Rate 
Round  Trip  Time 
Sustainable  Cell  Rate 
Subnetwork  Attachment  Point 
Transmission  Control  Protocol 
Transport  Service  Data  Unit 
User  Datagram  Protocol 
Usage  Parameter 
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VBR  Variable  Bit  Rate 

VCC  Virtual  Channel  Connection 

VCI  Virtual  Channel  Identifier 

VPI  Virtual  Path  Identifier 

WAN  Wide  Area  Network 
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Appendix  A:  One  TCP  bundled  with  several  UDP  Connections  on 
trans-Atlantic  ATM 


UDP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

fKbytes/s] 

TCP 

Throughput 

fKbytes/s] 

TCP 

Throughput 

fKbytes/s] 

l 

44.34 

79.2 

158.55 

160.68 

2 

44.48 

79.77 

146.45 

147.95 

3 

45.13 

80.96 

136.19 

137.1 

4 

43.6 

80.95 

127.8 

6 

44.76 

71.76 

115.4 

8 

42.53 

54.69 

101.89 

107.45 

12 

29.67 

53.58 

84.96 

100.33 

16 

27.99 

42.32 

79.71 

99.5 

Table  5 


UDP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 

1 6  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

fKbvtes/s] 

TCP 

Throughput 

fKbytes/s] 

l 

37.84 

79.84 

158.25 

160.04 

2 

37.62 

72.94 

146.82 

147.26 

3 

37.12 

136.48 

136.8 

4 

36.97 

72.2 

127.1 

127.99 

6 

36.83 

61.53 

112.42 

115.52 

8 

33.59 

50.26 

102.34 

108.19 

12 

25.12 

43.59 

85.35 

103.53 

16 

22.96 

37.32 

103.21 

Table  6 
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UDP  TCP  TSDU  LENGTH  =  10  Kbytes 


Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/sl 

TCP 

Throughput 

[Kbvtes/s] 

l 

52.58 

94.06 

160.05 

160.2 

2 

52.83 

92.71 

147.52 

146.91 

3 

52.89 

90.69 

136.56 

136.83 

4 

52.96 

89.34 

127.38 

127.59 

6 

52.77 

75.93 

114.62 

8 

44.84 

61.19 

1 

12 

E^33.9fl 

58.61 

85.19 

98.02 

16 

33.13 

45.9 

82.86 

97.04 

Table  7 


UDP  TCP  TSDU  LENGTH  =  20  Kbytes 


Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/s] 

l 

54.74 

99.76 

159.88 

160.7 

2 

54.6 

96.25 

147.87 

147.68 

3 

54.04 

95.67 

137.26 

4 

54.11 

92.19 

127.98 

6 

53.62 

76.91 

112.68 

115.61 

8 

45.31 

62.81 

103.33 

HES3EHI 

12 

34.86 

60.35 

86.15 

■ 

16 

33.82 

47.02 

83.13 

101.99 

Table  8 
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UDP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

l 

7.79 

7.79 

7.79 

7.79 

2 

7.79 

7.79 

7.79 

7.79 

3 

7.79 

7.79 

7.79 

7.79 

4 

7.79 

7.78 

7.79 

7.78 

6 

7.78 

7.78 

7.78 

6.91 

8 

7.78 

7.78 

6.76 

6.1 

12 

6.8 

6.46 

5.79 

5.13 

16 

6.1 

5.77 

5.05 

4.59 

Table  9 


UDP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 

50  Kbvtes 

_ _ - 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

l 

7.8 

7.8 

7.79 

7.79 

2 

7.79 

7.8 

7.79 

7.79 

3 

7.8 

7.79 

7.79 

7.78  1 

4 

7.8 

7.79 

7.79 

7.78 

6 

7.79 

7.79 

7.78 

6.72 

8 

7.79 

7.41 

6.79 

12 

6.41 

5.88 

5.77 

16 

5.46 

5.14 

5.03 

4.56 

Table  10 
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UDP  TCP  TSDU  LENGTH  =  1 0  Kbytes 

Bundles  TCP  Window  [  TCP  Window  TCP  Window  I  TCP  Window 
8  Kbvtes  16  Kbytes  32  Kbytes  50  Kbytes 


Mean  UDP 

Mean  UDP 

Mean  UDP 

Mean  UDP 

Throughput 

Throughput 

Throughput 

Throughput 

[Kbytes/s] 

[Kbvtes/s] 

[Kbytes/s] 

[Kbvtes/s] 

1 

7.79 

7.79 

7.79 

7.79 

2 

7.79 

7.79 

7.79 

7.79 

3 

7.79 

7.78 

7.79 

7.78 

4 

7.79 

7.78 

7.78 

7.78 

6 

7.78 

7.78 

7.78 

7.05 

8 

7.77 

7.76 

6.73 

6.23 

12 

6.78 

6.4 

5.69 

5.25 

16 

6.06 

5.75 

4.98 

4.67 

Table  11 

UDP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

TCP  Window 

TCP  Window 

TCP  Window 

8  Kbytes 

16  Kbytes 

32  Kbytes 

50  Kbytes 

Mean  UDP 

Mean  UDP 

Mean  UDP 

Mean  UDP 

Throughput 

Throughput 

Throughput 

Throughput 

[Kbytes/s] 

[Kbvtes/s] 

[Kbvtes/s] 

[Kbytes/s] 

l 

7.79 

7.79 

7.79 

7.79 

2 

7.79 

7.79 

7.79 

7.79 

3 

7.79 

7.79 

7.79 

7.78 

4 

7.79 

7.78 

7.78 

7.78 

6 

7.78 

7.78 

7.76 

6.82 

8 

7.78 

7.71 

6.69 

12 

6.78 

6.4 

5.74 

5.11 

16 

6.04 

5.75 

4.97 

4.58 

Table  12 
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UDP 

Bundles 


TCP  TSDU  LENGTH  =  4  Kbytes 


TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 

50  Kbvtes 

_ _ _ * _ 

Mean  UDP 
Loss  Rate 

f%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 

r%i 

Mean  UDP 
Loss  Rate 
[%] 

0 

0 

0 

0 

0.13 

Table  13 


0.17 

0.18 

TCP  TSDU  LENGTH  =  8  Kbytes 


TCP  Window 
1 6  Kbytes 


TCP  Window 
32  Kbytes 


TCP  Window 
50  Kbytes 
Mean  UDP 
Loss  Rate 
[%1 


UDP 

Bundles 


TCP  TSDU  LENGTH  =  10  Kbytes 

TCP  Window  I  TCP  Window  I  TCP  Window  [  TCP  Window 

8  Kbytes _ 16  Kbytes  32  Kbytes  50  Kbytes 

Mean  UDP  Mean  UDP  Mean  UDP  Mean  UDP 

Loss  Rate  Loss  Rate  Loss  Rate  Loss  Rate 

[%]  [%)  [%] _ [%] 


J_ 

2_ 

J3_ 

4_ 

6_ 

_8_ 

12 

16 


0 

0 

0.01 

0.04 

0.12 

0.07 

0.23 

0.21 


0 

0 

0.02 

0.02 

0.11 

0.12 

0.17 

0.11 


0 

0.02 

0.05 

0.04 

0.13 

0.17 

0.21 

0.14 


0 

0 

0.01 

0.05 

0.12 

0.13 

0.19 

0.21 


Table  15 


UDP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 

50  Kbytes 

Mean  UDP 
Loss  Rate 
\%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%1 

l 

0 

0 

0 

0 

2 

0 

0.01 

0 

0 

3 

| 

0.01 

4 

| 

6 

0.12 

0.09 

| 

8 

0.12 

0.14 

12 

■ 

0.17 

0.17 

16 

0.13 

0.2 

0.53 

0.53 

Table  16 
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Appendix  B:  One  UDP  bundled  with  several  TCP  connections  on 
trans-Atlantic  ATM 


TCP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 

16  Kbytes 

TCP  Window 

32  Kbytes 

TCP  Window 

50  Kbytes 

Mean  TCP 
Throughput 
[Kbvtes/sl 

Mean  TCP 
Throughput 
[Kbytes/sl 

Mean  TCP 
Throughput 
[Kbytes/sl 

Mean  TCP 
Throughput 
[Kbvtes/sl 

l 

64.63 

65.17 

160.18 

2 

41.35  1 

45.32 

44.63 

43.28 

3 

32.15 

18.94 

23.09 

21.2 

4 

21.12 

14.01 

13.73 

13.66  1 

6 

10.27 

9.95 

8.74 

8 

6.54 

5.94 

6.29 

6.49 

10 

4.62 

4.53 

4.66 

4.87 

Table  17 


TCP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 

16  Kbytes 

TCP  Window 
32  Kbvtes 

k - - — 

TCP  Window 

50  Kbytes 

Mean  TCP 
Throughput 
[Kbvtes/sl 

Mean  TCP 
Throughput 
[Kbytes/sl 

Mean  TCP 
Throughput 
[Kbytes/sl 

Mean  TCP 
Throughput 
^[Khvtes/s^^ 

l 

41.35 

73.18 

94.18 

2 

36.41 

62.98  1 

44.69 

44.41 

3  1 

31.05 

29.85 

29.75 

4 

22.56 

14 

15.9 

15.77 

6 

11.24 

8.22 

8.52 

9.18 

8 

7.28 

6.16 

7.7 

6.69 

10 

4.55 

5.58 

5.66 

Table  18 
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TCP 

TCP  TSDU  LENGTH  =  10  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  TCP 
Throughput 

[Kbvtes/s] 

Mean  TCP 
Throughput 
[Kbvtes/s] 

Mean  TCP 
Throughput 

[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

l 

39.16 

90.15 

153.5 

2 

39.51 

78.89 

59.06 

3 

33.65 

24.5 

19.37 

20.55 

4 

24.36 

11.87 

14.51 

13.63 

6 

11.07 

6.98 

9.09 

9.03 

8 

6.77 

5.61 

6.13 

6.79 

10 

4.38 

4.39 

4.8 

6.12 

Table  19 


TCP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  TCP 
Throughput 

[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbvtes/s] 

Mean  TCP 
Throughput 

[Kbvtes/s] 

l 

53.08 

93.91 

79.17 

77.97 

2 

52.46 

43.2 

32.8 

31.33 

3 

28.74 

16.85 

18.48 

18.77 

4 

11.41 

13.36 

13.57 

6 

9.5 

7.52 

8.76 

9.02 

8 

4.28 

5.94 

5.56 

6.31 

10 

4.54 

4.34 

4.49 

4.54 

Table  20 
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TCP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 

50  Kbytes 

_ * - 

UDP 

Throughput 

[Kbytes/sl 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

l 

7.78 

7.8 

7.78 

7.79 

2 

7.79  1 

7.79 

7.79 

7.78 

3 

7.79 

7.79 

6.88 

6.68 

4 

7.79 

7.53 

5.98 

5.96 

6 

7.79 

5.15 

5.37 

4.99 

8 

6.8 

4.75 

4.7 

4.67 

10 

5.48 

3.77 

3.75 

3.92 

Table  21 


TCP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/sl 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/sl 

UDP 

Throughput 

[Kbytes/s] 

l 

7.77 

7.79 

7.79 

7.79 

2 

7.79 

7.8 

7.79 

7.79 

3 

7.79 

7.79 

7.79 

7.79 

4 

7.79 

7.49 

5.91 

5.8 

6 

7.79 

5.28 

5.23 

5.3 

8 

7.69 

4.83 

4.69 

4.76 

10 

5.76 

3.83 

4 

4.32 

Table  22 
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TCP 

TCP  TSDU  LENGTH  =  10  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbvtes/s] 

UDP 

Throughput 

[Kbvtes/s] 

UDP 

Throughput 

[Kbvtes/s] 

l 

7.79 

7.8 

7.78 

7.79 

2 

7.78 

7.79 

7.79 

7.79 

3 

7.79 

7.79 

6.06 

5.72 

4 

7.78 

6.24 

5.33 

5.01 

6 

7.78 

4.38 

3.95 

3.98 

8 

7 

3.93 

3.9 

3.89 

10 

5.1 

3.85 

3.8 

3.92 
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TCP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbvtes/s] 

UDP 

Throughput 

[Kbytes/s] 

l 

7.79 

7.79 

7.79 

7.78 

2 

7.78 

7.8 

7.78 

7.69 

3 

7.79 

7.79 

6.23 

6.25 

4 

7.79 

6.09 

5.42 

5.36 

6 

7.78 

5.07 

4.66 

4.67 

8 

5.18 

4.02 

3.88 

3.77 

10 

5.18 

3.81 

3.89 

3.77 
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Appendix  C:  UDP  bundled  connections  on  trans-Atlantic  ATM 


UDP  Bundles 

PCR 

2  Mbps 

UDP  Throughput 
[Kbvtes/s] 

1 

7.8 

2 

7.79 

3 

7.79 

4 

7.79 

6 

7.8 

8 

7.8 

10 

7.79 

12 

7.78 

14 

4.11 

16 

3.57 

Table  25 


UDP  Bundles 

PCR 

2  Mbps 

Mean  UDP  Loss 
Rate 
\%] 

1 

0 

2 

0 

3 

0.02 

4 

0.02 

6 

0.05 

8 

0.09 

10 

0.18 

12 

0.24 

14 

0.22 

16 

0.34 

Table  26 
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Appendix  D:  One  TCP  bundled  with  several  UDP  Connections  on 
LAN  ATM 


UDP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

^[Kbytes/s^^ 

l 

105.42 

140.81 

139.22 

2 

99.68 

129.18 

128.1 

128.52 

3 

99.28 

118.93 

117.82 

118.93 

4 

94.94 

1 10.89 

109.89 

6 

79.32 

96.85 

95.47 

8 

65.25 

86.44 

87.44 

94.27 

10 

58.16 

78.08 

79.98 

90.91 

Table  27 


UDP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

^^H^b}4es/s^^ 

l 

79.76 

140.97 

139.4 

2 

79.75 

129.34 

128.45 

128.24 

3 

79.76 

119.1 

4 

79.7 

i 

110.82 

i 

6 

72.49 

96.8 

95.9 

8 

54.63 

84.23 

88.35 

94.89 

10 

47.09 

76.87 

93.72 

Table  28 
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UDP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

[Kbvtes/s] 

TCP 

Throughput 

[Kbytes/s] 

TCP 

Throughput 

[Kbytes/s] 

l 

140.35 

140.68 

140.42 

141.21 

2 

128.72 

129.12 

129.02 

128.86 

3 

118.76 

119.15 

119.25 

119.46 

4 

110.78 

110.73 

111.01 

6 

95.88 

96.68 

97.06 

100.94 

8 

84.9 

86 

89.61 

94.61 

10 

77.02 

78.42 

81.42 

92.4 

Table  29 


UDP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbvtes/sl 

Mean  UDP 
Throughput 
[Kbytes/s] 

l 

7.79 

7.79 

7.79 

7.79 

2 

7.78 

7.79 

7.79 

7.79 

3 

7.78 

7.79 

7.79 

7.78 

4 

7.78 

7.78 

7.78 

7.78 

6 

7.77 

7.77 

7.72 

6.67 

8 

7.73 

7.75 

6.62 

5.95 

10 

7.31 

6.85 

6.13 

5.36 

Table  30 
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UDP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 

[Kbvtes/s] 

1 

7.79 

7.79 

7.79 

7.78 

2 

7.79 

7.78 

7.78 

7.79 

3 

7.78 

7.78 

7.78 

7.78 

4 

7.78 

7.78 

7.78 

7.77 

6 

7.77 

7.77 

7.58 

6.61 

8 

7.75 

7.5 

6.63 

5.9 

10 

7.18 

6.73 

6.1 

5.37 

Table  31 


UDP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbvtes/s] 

Mean  UDP 
Throughput 
[Kbytes/s] 

Mean  UDP 
Throughput 
[Kbytes/s] 

l 

7.79 

7.78 

7.79 

7.79 

2 

7.79 

7.78 

7.79 

7.78 

3 

7.78 

7.78 

7.79 

7.78 

4 

7.78 

7.78 

7.78 

7.78 

6 

7.78 

7.77 

7.6 

6.6 

8 

7.75 

7.66 

6.61 

5.92 

10 

7.32 

6.84 

6.07 

5.37 

Table  32 


UDP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 

[%1 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 

r%] 

l 

0.008 

0.031 

0.028 

2 

hebh 

0.022 

3 

0.011 

0.029 

4 

0.064 

0.051 

6 

0.08 

8 

0.176 

■ 

10 

0.255 

0.139 

0.13 

0.234 

Table  33 


UDP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Loss  Rate 

r%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

l 

hheezshi 

0.011 

| 

0.042 

2 

0.057 

| 

0.022 

3 

0.091 

0.055 

4 

0.065 

0.091 

6 

■ 

0.062 

0.098 

8 

0.167 

0.138 

0.083 

0.07 

10 

0.2 

0.21 

0.083 

0.076 

Table  34 
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UDP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

Mean  UDP 
Loss  Rate 
[%] 

l 

0.008 

HKE2HM 

0.044 

0.036 

2 

0.029 

0.014 

0.04 

3 

0.049 

0.019 

0.029 

4 

6 

■ 

| 

0.111 

8 

0.116 

0.117 

0.088 

0.07 

10 

0.213 

0.113 

0.116 

0.071 

Table  35 
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Appendix  E:  One  UDP  bundled  with  several  TCP  connections  on 
LAN  ATM 


TCP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  TCP 
Throughput 
[Kbvtes/sl 

Mean  TCP 
Throughput 
[Kbvtes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 

[Kbytes/s] 

l 

103.5 

140.71 

138.9 

139.65 

2 

69.27 

69.8 

70.57 

3 

45.55 

46.03 

46.29 

46.98 

4 

34.24 

34.54 

34.88 

34.19 

6 

22.49 

22.74 

22.81 

24.19 

8 

16.84 

17.35 

17.67 

18.25 

10 

13.54 

13.7 

14.25 

15.23 

Table  36 


TCP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  TCP 
Throughput 
[Kbvtes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 

[Kbytes/s] 

l 

79.65 

138.87 

139.67 

2 

70.43 

69.34 

69.81 

70.19  1 

3 

46.06 

46.4 

46.78 

4 

33.18 

34.53 

35.05 

34.14 

6 

21.91 

22.86 

22.8 

23.88 

8 

16.37 

17.2 

17.85 

18.97 

10 

13.05 

13.71 

14.07 

15.14 

Table  37 
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TCP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbvtes/s] 

Mean  TCP 
Throughput 
[Kbytes/s] 

Mean  TCP 
Throughput 
[Kbvtes/s] 

l 

140.28 

140.54 

140.25 

140.81 

2 

68.79 

70.24 

70.29 

70.14 

3 

46.26 

46.89 

46.53 

46.55 

4 

34.3 

34.58 

34.67 

33.73 

6 

22.71 

22.75 

22.48 

23.22 

8 

17.07 

16.94 

17.2 

18.61 

10 

13.69 

13.2 

14.62 

14.83 

Table  38 


TCP 

TCP  TSDU  LENGTH  =  4  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

l 

7.79 

7.79 

7.79 

7.79 

2 

7.79 

7.78 

7.78 

7.78 

3 

7.78 

7.78 

7.78 

7.73 

4 

7.78 

7.77 

7.77 

5.83 

6 

7.78 

7.71 

6.21 

5.11 

8 

7.77 

7.63 

5.38 

4.7 

10 

7.77 

7.16 

5.03 

3.99 

Table  39 
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TCP 

TCP  TSDU  LENGTH  =  8  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

fKbytes/s] 

l 

7.79 

7.79 

7.79 

7.78 

2 

7.78 

7.78 

7.79 

7.78 

3 

7.79 

7.78 

7.79 

7.56 

4 

7.77 

7.79 

7.75 

5.84 

6 

7.78 

7.72 

6.22 

5 

8 

7.77 

7.57 

5.46 

4.03 

10 

7.73 

7.16 

5.02 

3.97 

Table  40 


TCP 

TCP  TSDU  LENGTH  =  20  Kbytes 

Bundles 

TCP  Window 

8  Kbytes 

TCP  Window 
16  Kbytes 

TCP  Window 
32  Kbytes 

TCP  Window 
50  Kbytes 

UDP 

Throughput 

[Kbytes/s] 

UDP 

Throughput 

fKbytes/s] 

UDP 

Throughput 

[Kbvtes/s] 

UDP 

Throughput 

fKbytes/s] 

l 

7.79 

7.78 

7.79 

7.79 

2 

7.78 

7.79 

7.78 

7.78 

3 

7.79 

7.79 

7.78 

7.27 

4 

7.78 

7.76 

7.69 

5.68 

6 

7.78 

7.62 

5.67 

4.94 

8 

7.77 

7.46 

3.94 

10 

7.76 

6.28 

4.08 

3.9 

Table  41 
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Appendix  F.  UDP  bundled  connections  on  LAN  ATM 


UDP  Bundles 

PCR 

2  Mbps 

UDP  Throughput 

[Kbytes/s] 

1 

7.79 

2 

7.79 

3 

7.78 

4 

7.75 

6 

7.77 

8 

7.77 

10 

7.76 

12 

7.15 

14 

6.69 

16 

6.46 

18 

6.03 

20 

5.78 

26 

5.17 

Table  42 
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UDP  Bundles 

PCR 

2  Mbps 

Mean  UDP  Loss 
Rate 
[%] 

1 

0.044 

2 

0.13 

3 

4 

0.103 

6 

0.15 

8 

0.19 

10 

0.255 

12 

0.329 

14 

16 

18 

0.26 

20 

0.253 

26 

0.302 

Table  43 
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